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Abstract 

The  services  within  the  Military  Healthcare  System  have  pursued  metric  set 
development  to  measure  established  goals  and  perceived  information  requirements.  The 
Assistant  Secretary  of  Defense  for  Health  Affairs  charged  the  TMA  to  streamline  metric 
sets  within  the  MHS  and  create  a  standard  metric  set.  This  study  was  initiated  to  validate 
the  proposed  standard  metric  set  and  solicit  input  from  the  field.  In  addition,  the  set  was 
compared  to  performance  measurement  tools  used  in  healthcare  organizations  in  the 
private  sector.  A  three-part  questionnaire  submitted  to  respondents  within  the  MHS 
exhibited  no  significant  differences  in  opinion  as  to  general  strategic  concepts  and  the 
utility  of  the  standard  set  between  services,  professions,  and  levels  of  command.  Of  the 
47  measures  proposed,  only  six  exhibited  any  significant  difference  in  opinion.  The 
differences  were  mainly  between  professions  and  levels  of  command.  More  interestingly 
were  the  similarities  of  opinion  among  the  respondents.  Satisfaction  measures  were 
highly  rated  with  little  variation.  A  Factor  Analysis  compared  TMA’s  assigned  domains 
for  each  measure  to  the  underlying  constructs  exhibited  in  the  completed  questionnaires. 
The  results  portrayed  opportunities  for  perverse  incentives,  unintended  behavior  changes, 
and  improper  categorization.  A  comparison  with  the  private  sector  demonstrated  issues 
pertaining  to  domains  and  possible  metric  additions.  Also,  metric  sets  at  health  plans 
were  the  most  similar  to  the  MHS.  A  high  level  of  agreement  exists  among  the 
respondents  as  to  the  need  to  streamline  the  metric  set  system  and  the  proposed  set’s 
ability  to  meet  the  decision-making  requirements  of  the  MHS’s  healthcare  leaders. 
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An  Analysis  and  Validation  of  the  Executive  Level  Standard  Metric  Set 
Created  at  the  TRICARE  Management  Activity 

Introduction 

The  Military  Health  System  (MHS)  is  saturated  with  performance  measures,  also 
known  as  metrics,  to  measure  a  number  of  processes  and  outcomes.  Each  service  has 
encouraged  the  creation  of  metrics  to  measure  what  respective  executive  medical  leaders 
deemed  important  enough  to  track.  In  addition,  within  each  service  several  layers  of 
command,  the  regions.  Military  Treatment  Facility  (MTFs),  and  departments,  have  all 
created  their  own  set  of  performance  measures.  Further  more,  with  the  creation  of 
organizations  such  as  the  TRICARE  Management  Activity  (TMA),  The  Defense  Medical 
Oversight  Committee  (DMOC),  and  other  DoD  proponents;  additional  metrics  to 
measure  performance  were  produced  to  track  processes  and  health  care  outcomes. 
Finally,  the  magnitude  of  the  system  further  complicates  metric  preparation.  The  MHS  is 
composed  of  465  MTFs  and  over  eight  million  beneficiaries  (U.S.  GAO,  1999). 

During  the  last  five  years  the  MHS  has  struggled  with  a  growing  number  of 
reporting  requirements.  Thus,  TMA  devised  an  executive  level  standard  metric  set  in  an 
attempt  to  consolidate  critical  measurement  components  under  one  umbrella.  The 
leadership  at  TMA  (Executive  Director,  Deputy  Executive  Director,  and  the  Chief  of 
Staff)  and  the  Acting  Assistant  Secretary  of  Defense  for  Health  Affairs  have  encouraged 
a  review  of  various  metric  arrays  and  the  construction  of  one  standard  set. 

This  was  accomplished  for  a  number  of  reasons.  First,  to  insure  that  the 
corporation  is  measuring  what  is  important  based  on  selected  criteria.  Second,  to 
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minimize  the  number  of  reports  submitted  by  the  lead  agent  regions  and  health  care 
activities.  Third,  to  streamline  the  number  of  independent  metric  sets,  at  the  executive 
level,  into  one  standard  set.  Fourth,  to  maximize  data  collection  efforts  by  reducing  the 
number  of  times  the  mainframe  is  accessed.  Fifth,  to  reduce  the  organization’s  time  lag 
between  execution,  data  collection,  data  processing,  and  reporting.  Sixth,  to  develop  an 
organizational  framework  that  recognizes  the  different  levels  of  management  oversight 
and  operational  process  requirements.  Seventh,  to  enable  accurate  and  timely  feedback  to 
the  executive  level  and  the  operational  echelons. 

These  seven  major  reasons  create  the  impetus  and  set  the  boundaries  for  the  new 
metric  set.  TMA  set  up  a  working  group  with  representation  from  the  three  services  to 
create  a  proposed  standard  metric  set  based  on  existing  measures.  After  the  initial  design, 
the  next  logical  step  in  the  process  is  to  assess  the  set  for  applicability  in  decision  making 
at  each  MTF,  region,  and  executive  staff.  Now,  that  the  TMA  and  Health  Affairs  (HA) 
metrics-working  group  has  fashioned  this  set,  the  enterprise  will  validate  the  set  and 
solicit  input  from  the  regional  and  MTF  command  teams  and  the  MHS  executives.  Two 
methods  are  used.  First,  to  circulate  the  set  among  selected  lead  agent  regions,  MTF 
command  teams,  and  executives  to  solicit  feedback  in  the  form  of  a  questionnaire.  This 
feedback  would  include  recommendations  for  additional  metrics  and  responses  to  general 
metric  and  strategic  implementation  statements.  Second,  to  compare  the  proposed 
standard  set  against  a  number  of  civilian  organizations,  to  determine  the  gap  between  our 


measures  and  theirs. 
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Background 

Performance  measures  are  normally  implemented  to  drive  a  change  in  behavior. 
They  measure  performance  and  outcomes  within  an  enterprise  to  meet  stated 
organizational  goals.  To  accomplish  this,  the  organization’s  strategic  plan  is  defined 
operationally.  Then  organizational  goals  or  management  objectives  are  determined  within 
the  organization’s  strategic  plan.  The  selection  of  performance  measures  should  be 
mapped  back  from  the  operational  venue  to  the  strategic  plan  based  on  those 
organizational  goals.  This  enables  executive  leaders  to  steer  the  organization.  The  link 
between  measures  and  strategic  plan  is  important  to  ensure  that  the  organization  is 
accomplishing  those  goals. 

This  paper  will  concentrate  on  those  initiatives  at  the  Department  of  Defense 
(DoD)  level  (includes  TMA,  OSD-HA,  and  the  DMOC).  Discounting  the  various  metric 
sets/dashboards  created  by  the  services,  TMA  is  the  proponent  for  17  separate  sets 
(Appendix  A).  Five  of  these  are  the  primary  performance  measurement  tools  for  the 
MHS.  A  comparison  of  the  five  sets  is  attached  (Appendix  B).  To  better  understand  why 
the  MHS  is  so  fragmented  in  its  effort  of  develop  effective  measurement  tools  and  in 
implementing  a  strategic  plan  with  defined  operational  goals,  a  short  description  of  the 
MHS  organization  and  the  decision  making  process  is  presented. 

The  MHS  has  a  number  of  key  stakeholders.  They  include:  The  members  of 
Congress,  the  Under  Secretary  of  Defense  for  Personnel  and  Readiness  (USD  P&R),  the 
DMOC,  the  Assistant  Secretary  of  Defense  for  Health  Affairs  (ASD  HA),  the  Executive 
Director  of  TMA,  the  Surgeons  General  of  each  service  (SGs),  the  beneficiaries,  and  the 
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providers/employees.  There  are  others,  but  for  discussion  purposes  from  a  policy 
perspective,  these  are  the  main  ones. 

In  the  military,  decision  making  normally  follows  a  published  chain  of  command. 
For  example,  if  the  boss  says  to  do  something,  it  is  done.  The  present  structure  in  the 
MHS  though,  does  not  exhibit  a  simple  chain  of  command.  This  phenomenon  is  due  in 
most  part  to  the  department’s  two  missions,  readiness  and  benefits  management.  Both 
these  missions,  though  perceived  to  be  discrete,  overlap.  The  peacetime  elements  of  care 
contribute  to  readiness  (wellness)  and  the  active  duty  personnel  that  provide  care  in 
MTFs  also  deploy  (RAND  Report,  2001).  Though  both  these  missions  share  common 
ground,  “joint  production  of  readiness  and  benefits  involves  a  complicated  set  of  trade¬ 
offs  and  management  challenges”  at  all  levels  of  command/authority  (RAND  Report). 

There  are  numerous  leaders  and  agencies  that  impact  those  healthcare  facilities 
and  regions  with  business  decisions,  rules,  and  requirements.  In  some  cases,  the 
organizations  with  responsibility  for  a  program  may  not  have  the  authority  to  execute  it. 
Thus,  decision  making  in  the  MHS  has  evolved  into  a  complex  dance  incorporating 
consensus  building,  collaboration,  excessive  coordination  needs,  and  negotiation.  The 
MHS  is  made  up  of  three  services  (represented  by  the  respective  SGs),  TMA  (a  DoD 
agency),  and  the  office  of  the  ASD  (HA).  Four  of  the  five  organizations  work  for 
different  entities.  This  creates  serious  challenges  in  designing  and  implementing  viable 
metric  sets  to  monitor  organizational  performance  based  on  management  objectives. 

The  SGs  answer  to  their  Chiefs  of  Staff,  who  in  turn  report  to  their  respective 
service  (Army,  Navy,  Air  Force)  Secretaries.  The  Executive  Director  of  TMA,  charged 
with  the  responsibility  of  insuring  the  seamless  provision  of  peacetime  health  care  within 
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the  system,  reports  to  the  ASD  (HA),  who  reports  to  the  USD  (P&R).  This  organizational 
structure  does  not  encourage  unity  of  effort.  The  greatest  potential  for  conflict  arises 
between  TMA  and  the  SGs.  In  an  attempt  to  improve  the  decision-making  process,  the 
DoD  created  the  DMOC.  The  DMOC  functions  as  the  DoD  executive  level  group 
responsible  for  overseeing  various  issues  confronting  the  MHS.  It  is  composed  of  the 
USD  (P&R),  selected  service  secretaries,  the  service  Vice  Chiefs  of  Staff,  and  the  ASD 
(HA).  This  committee  was  designed  to  provide  oversight  and  direction  to  the  MHS  and 
direct  those  efforts  from  one  overarching  group  of  executive  leaders.  They  were  created 
in  response  to  the  ever-increasing  costs  of  delivering  healthcare  in  the  MHS.  They  were 
charged  to  study  options  to  decrease  those  costs  and  recommend  organization  wide 
business  efficiencies. 

Even  with  the  creation  of  the  DMOC,  on  31  August  1999,  the  challenges  in  the 
MHS  lie  in  decision-making.  A  stove  piped  organization  that  encourages  parochialism 
among  its  services  will  undoubtedly  steer  itself  in  at  least  four  different  directions  (the 
Army,  Air  Force,  Navy,  and  OSD  (HA)).  The  resources,  personnel,  funding,  facilities, 
and  equipment  are  under  the  purview  of  the  services,  title  10  of  the  U.S.  Code.  The  policy 
making,  operational  planning,  and  in  many  cases  direction  to  execute  lie  with  the  DoD 
elements.  Therefore,  leaders  in  the  Defense  Health  Plan  (DHP),  composed  of  OSD  (HA) 
and  TMA  are  faced  with  decision-making  in  a  cooperative  environment.  All  major  policy 
and  operational  decisions  to  include  their  execution  must  be  collectively  decided  among 
the  TMA,  the  SGs  and  OSD  (HA).  Proper  buy-in  is  key  to  insure  the  success  of  a 
program  implemented  in  support  of  our  beneficiaries  and  Congressional  mandates.  To 
accomplish  the  MHS/DHP  mission  TMA  and  the  SGs  work  hand  in  hand  on  all 
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optimization  plan  programs  and  initiatives.  By  creating  Integrated  Project  Teams  (with 
service  representation),  the  TMA  is  pursuing  a  successful  collective  decision  making 
process. 

In  1998  and  1999,  the  TMA  developed  the  optimization  plan  -  a  strategic  vision 
for  the  MHS,  both  for  the  direct  care  and  purchased  care  side.  This  plan  has  not 
permeated  throughout  the  entire  MHS,  due  to  its  complexity  and  difficulty  in 
coordinating  implementation  throughout  the  system.  This  is  an  example  of  a  collective 
decision  that  did  not  fully  succeed.  It  is  difficult  for  this  corporation  to  define  a  single 
vision,  mission,  a  set  of  strategic  goals,  and  adequate  performance  measures  to  monitor 
continuous  improvement.  Unfortunately,  in  this  case  the  optimization  plan  is  a  critical 
component  to  fully  integrate  the  MHS  by  implementing  key  models  to  improve  wellness, 
readiness,  resource  allocation,  enrollment  and  metrics. 

Thus,  the  system  is  still  faced  with  an  inordinate  number  of  metric  sets  to  measure 
outcomes  and  process  performance  for  each  agency’s  (Army,  Navy,  Air  Force)  medical 
service  and  the  DHP.  The  SGs  have  their  own  metric  sets  that  map  to  their  “strategic 
plan”  and  vision,  while  TMA  has  some  to  support  three  requirements:  legislative 
mandates,  the  optimization  plan,  and  DMOC  directives.  In  an  endeavor  to  consolidate 
some  of  these  measures  at  the  executive  level,  the  ASD  (HA)  and  the  DMOC  have 
challenged  TMA  and  the  services  to: 

1.  Identify  all  current  MHS  created  metric  sets. 

2.  Identify  externally  directed  metric  sets. 

3.  Develop  a  metric  framework  that  reflects  the  different  levels  of  management. 
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4.  Make  recommendations  for  implementing  an  integrated,  standardized,  and 
simplified  measurement  and  management  strategy. 

To  accomplish  this  task,  TMA  (Health  Program  Analysis  and  Evaluation  department) 
stood  up  a  Tri-service  Metrics  Working  Group.  The  goal  of  this  group  is  to  consolidate 
all  similar  metric  sets,  streamline  the  process,  and  create  a  system  similar  to  that  in  the 
private  sector  using  best  practice  guidelines.  Their  first  step  was  to  create  a  proposed 
MHS  wide  standard  metric  set  (Appendix  C). 

Therefore,  this  paper  concentrates  on  comparing  the  opinions  of  the  different 
services,  professions,  and  levels  of  command/authority  regarding  metric  set  design, 
strategy,  and  implementation.  It  also  reviews  the  importance  and  applicability  of  the 
proposed  standard  measures,  based  on  a  representative  sample  of  respondents.  In 
addition,  the  gap  analysis  between  the  proposed  standard  set  and  selected  private  sector 
health  care  systems  and  plans  identifies  salient  differences  in  performance  measurement 
areas/domains  and  specific  measures.  These  comparisons  will  assist  decision-makers  and 
experts  in  identifying  any  differences  or  agreements  in  opinion  within  the  MHS  and 
external  to  the  public  sector. 


Literature  Review 

Performance  measures  are  derived  from  an  organization’s  strategy  and  selected 
business  factors  that  assist  an  organization’s  leaders  in  achieving  its  vision  (Brown, 

1996).  Metrics  or  performance  measures  are  a  key  component  that  enables  an 
organization  to  meet  its  strategic  goals  and  objectives.  According  to  Kaplan  and  Norton 
(1996),  an  organization  must  be  able  to  define  corporate  objectives  and  selected  measures 
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to  achieve  said  objectives.  “  A  common  phrase  today  is  ‘you  can’t  manage  what  you 
can’t  measure’”  (Jennings  &  Staggers,  1999,  p.  17).  Measures  should  be  relevant, 
reliable,  clear,  and  tested  to  insure  data  availability  to  compute  those  measures.  (Price 
Waterhouse,  1995).  In  addition,  with  the  changing  nature  of  business  in  today’s  corporate 
environment,  metrics  should  also  be  flexible.  Finally,  the  prevailing  literature 
recommends  that  metric  sets  should  reflect  the  organization’s  vision  and  values  (Schultz 
&  Bowers,  1997). 

The  fundamental  changes  to  the  nation’s  health  care  system  over  the  past  decade 
have  fueled  an  expansion  in  the  production  of  performance  measures  or  performance 
dashboards  (Berkow  &  Seth,  2000).  The  greatest  driver  for  the  creation  of  metrics  was 
cost.  As  the  cost  of  health  care  delivery  rose,  the  risk  of  deteriorating  fiscal  health  within 
health  care  organizations  increased.  Therefore,  these  organizations  initially  focused  on 
financial  measurements. 

Berkow  and  Seth  (2000)  point  out  that  the  fifth  most  common  reason  for 
corporate  crisis  was  inadequate  performance  measurement.  Crisis  was  defined  as 
“sudden,  dramatic  drops  in  both  total  margin  and  cash  flow...”  (Berkow  &  Seth,  p.  6). 
Based  on  the  outcome  many  organizations  concentrated  their  efforts  on  what  was 
perceived  to  be  the  symptom,  lack  of  revenue  and  high  expenses.  The  Health  Care 
Advisory  Board  has  examined  over  50  organizations  to  determine  what  makes  a 
successful  performance  measurement  system.  Their  results  revolve  around  four  elements, 
metric  balance,  metric  austerity,  graphic  display,  and  action  triggers  (Berkow  &  Seth). 
These  elements  are  central  to  reducing  the  possible  risk  of  organizational  crisis.  Metric 
austerity  reduces  the  number  of  performance  measures  to  approximately  30,  with  little 
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duplication.  Action  triggers  are  the  critical  thresholds,  similar  to  control  measures  that 
allow  management  to  quantify  a  position  and  a  course  of  action.  Most  importantly,  metric 
balance  ensures  a  broad-based  approach  to  performance  measurement  selection  from  key 
areas  (operational,  customer  support,  financial,  clinical  quality,  etc.).  Metric  balance 
introduces  those  areas  or  domains  to  identify  trends  and  outcomes  that  will  affect  the 
overall  health  of  the  organization.  The  TMA  has  determined  the  domains  to  be  quality, 
cost,  access,  health  outcomes,  and  readiness. 

As  quality  and  access  issues  reached  the  same  level  of  scrutiny  as  cost,  metric  sets 
expanded  to  measure  health  outcome  and  satisfaction  criteria.  In  addition  to  business 
innovations,  these  changes  were  based  on  external  demands  by  beneficiaries,  not-for- 
profit  organizations  (Joint  Commission  on  Accreditation  of  Healthcare  Organizations 
(JCAHO),  National  Committee  for  Quality  Assurance  (NCQA),  etc.,)  and  payers. 

JCAHO  states  that  a  metric  set  is  an  distribution  of  interrelated  measures 
reflecting  process,  outcomes,  or  both  that  enable  a  comparison  of  organizations 
longitudinally  both  internally  and  externally  (JCAHO,  1993).  Based  on  the  literature, 
metrics  are  tools  used  to  track  trends,  to  set  targets,  to  make  mid  course  corrections,  to 
validate  that  performance  is  commensurate  with  strategic  goals,  and  to  effect  continuous 
improvement  (Birchard,  1995,  Kaplan  and  Norton,  1993,  Braun,  Koss,  and  Loeb,  1999, 
Jennings  and  Staggers,  1999).  Metrics  should  not  only  measure  performance,  but  should 
contain  “predictive  power”  (Birchard,  1995,  p.  44).  For  example,  metrics  should  not  be 
passive,  but  should  capture  key  business  changes,  they  are  leading  indicators  to  help 
predict  future  organizational  success,  and  some  can  determine  the  value  of  company 
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assets  to  produce  wealth  in  the  future  (Birchard,  1995).  An  example  is  physician 
satisfaction,  which  often  leads  to  financial  health  (Berkow  &  Seth,  2000). 

According  to  Jennings  and  Staggers  (1999)  “businesses  translate  strategic 
objectives  into  a  coherent  set  of  performance  measures  to  define  and  communicate 
organizational  priorities  to  various  constituents  such  as  managers,  employees,  investors, 
and  customers”  (p.  18).  To  effectively  capture  and  communicate,  metrics  must  be 
properly  defined,  act  as  catalysts  to  change  behavior,  and  be  understood  throughout  the 
organization.  All  measures  should  be  evaluated  for  reliability  and  validity  (McGlynn, 
1998).  This  is  important  to  ensure  that  the  results  are  consistent  and  that  they  actually 
measure  the  required  concept.  According  to  Normand,  McNeil,  Peterson,  and  Palmer,  one 
can  solicit  information  from  a  pool  of  experts  to  determine  importance  or  appropriateness 
of  performance  indicators  (1998).  In  addition,  to  study  the  level  of  understanding  within 
an  organization,  “it  is  also  useful  to  draw  a  distinction  between  underlying  constructs 
versus  observed  variables”  (Hatcher,  1994).  Factor  Analysis  and  other  statistical  methods 
can  be  used  to  measure  understanding  and  applicability.  Therefore,  companies  migrating 
from  passive  fiscal  oversight  have  adopted  a  number  of  concepts  to  create,  define, 
implement,  and  sustain  their  dynamic  metric  sets. 

One  of  the  most  popular  is  Kaplan  and  Norton’s  Balanced  Score  Card  system. 
The  balanced  score  card  augments  financial  measures  with  performance  measures  in  the 
following  venues:  1)  a  company’s  relationship  with  its  customers,  2)  its  key  internal 
process,  and  3)  its  learning  and  growth  (Kaplan  &  Norton,  1996).  Companies  in  the 
private  sector  use  the  scorecard  methodology  to  assist  in  defining  parameters  to  insure 
that  metrics  are  not  created  in  a  haphazard  manner.  The  United  States  Army  Medical 
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Department  is  considering  implementing  the  balanced  scorecard  approach.  According  to 
Mr.  Ron  James,  PASBA,  “the  intent  of  such  an  approach  is  to  avoid  the  creation  of 
metrics  in  order  to  fill  a  perceived  void  in  information  and  to  try  to  coordinate  the 
development  of  metrics  among  all  segments  of  the  Service...”  (Metrics  Working  Group 
minutes,  February  28, 2001).  ‘The  goal  is  to  develop  a  consensus  on  metrics  and  their 
definition  within  the  Army  and  across  the  various  levels  within  the  chain-of-command” 
(Metrics  Working  Group  minutes). 

This  concept  enables  companies  to  maintain  a  manageable  number  of  metrics 
organization  wide  and  at  the  executive  level.  There  are  a  number  of  other  models  other 
than  the  balanced  score  card,  and  what  they  all  have  in  common  is  an  emphasis  on  tying 
performance  measures  to  management  objectives.  In  addition,  once  the  goals  or 
management  objectives  are  defined  and  linked  to  the  strategy,  Kaplan  and  Norton 
introduce  “strategy  maps”  to  guide  a  company  in  “converting  its  various  assets  into 
desired  outcomes”  (Kaplan  and  Norton,  2000,  p.  168). 

In  health  care,  not  only  is  it  important  to  be  able  to  map  the  objectives  and  goals 
to  strategy  and  vision;  it  is  critical  to  measure  performance  within  the  tenets  of  the  iron 
triangle  (Access,  Quality,  and  Cost)  and  within  Donabedian’s  framework  (Structure, 
Process,  Outcome).  The  TMA  proposed  standard  set  is  based  on  the  tenets  of  the  iron 
triangle  and  even  uses  them  as  the  key  performance  measurement  areas. 

A  balanced  score  card  approach  can  be  applied  to  each  of  the  tenets.  For  example, 
one  can  view  quality  from  at  least  four  different  perspectives:  the  clinical,  the  customers’, 
the  communities’  and  the  organizations’  (Clancy,  1997  and  Jennings  &  Staggers,  1997). 
Donabedian’s  model  is  a  superb  framework  to  use  when  deciding  what  one  is  trying  to 
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measure  on  the  continuum  of  health  care  provision.  It  is  important  to  point  out  that 
presently  health  care  leaders  are  measuring  nearly  exclusively  at  the  outcome  phase. 
McGlynn  (1998)  states  that  outcome  measures  “more  clearly  represent  the  goals  of  the 
healthcare  system”  (p.  474).  Previously  managers  measured  quality  at  the  process  phase 
to  identify  variation  in  performance.  But  with  the  onset  of  a  focus  on  value:  “the  best 
outcomes  for  the  resources  invested”  (Clancy,  1997,  p.  1567)  and  the  impetus  for  quality 
improvement,  outcomes  management  has  risen  to  the  forefront.  Outcomes  management  is 
the  balancing  of  cost  and  quality  by  improving  the  process  of  care  as  defined  by  the 
results  of  that  care  (Jennings  and  Staggers,  1997).  Therefore,  a  key  goal  in  healthcare  is 
to  examine  patient  outcomes  to  determine  efficiency  relative  to  processes  and  resources 
used.  (Jennings  and  Staggers).  By  measuring  performance  at  outcome  phase,  the  manager 
can  resolve  what  changes  need  to  be  made  in  the  structure  and  process  phases  to  improve 
quality  and  reduce  cost.  In  this  example,  metrics  should  be  catalysts  for  change  (non¬ 
passive)  and  should  drive  behaviors  towards  a  defined  and  understood  strategic  goal. 

An  important  aspect  of  strategy,  vision,  values  and  goals  is  a  common  proponent. 
The  literature  bespeaks  of  an  organizational  vision,  a  set  of  values,  and  one  overarching 
strategic  plan  with  specified  goals.  In  the  MHS,  four  such  systems  exist,  the  Air  force. 
Navy,  Army,  and  OSD  (HA).  One  CEO  or  common  proponent  does  not  exist;  therefore, 
each  service  and  the  DoD  tend  to  move  in  differing  directions.  The  perceived  and  real 
differences  in  culture,  values,  mission,  and  structure  affect  performance  measurement 
design  and  implementation. 

There  is  widespread  concern  that  the  number  of  different  metric  sets  and 
independent  initiatives  hinders  a  concerted  effort  at  unifying  goals  and  objectives.  An 
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example  of  different  initiatives  is  the  Army’s  drive  to  implement  the  balanced  score  card 
approach  to  designing  their  performance  measurement  system.  The  other  two  services  are 
not  pursuing  this  course,  but  have  concentrated  on  other  tools  and  on  interpreting  and 
implementing  the  optimization  plan  to  suit  their  service  priorities.  The  differing  missions 
and  readiness  priorities  also  add  to  the  strategic  goals  and  objectives  confusion.  TMA 
under  the  purview  of  OSD  (HA)  has  tried  to  combine  and  create  metric  sets  for  use  in  the 
field.  According  to  HPA&E  at  TMA,  these  efforts  are  met  with  varying  degrees  of 
enthusiasm.  Examples  include  TRICARE  Operational  Performance  Statement  (TOPS), 
Population  Health  Operational  Tracking  and  Optimization  (PHOTO)  System,  and  the 
MHS  Executive  Summary.  Though,  these  significant  differences  exist,  it  seems  based  on 
a  number  of  presentations  and  papers  available  at  the  services’  web  sites  that  the  main 
principle  is  improved  patient  care.  From  the  “brick  and  mortar”  perspective  there  may  be 
agreement  as  to  metric  selection  and  use. 

Therefore,  it  seems  appropriate  to  ask.  Will  the  proposed  standard  metric  set  be 
any  different?  Once  designed  and  fielded,  are  the  MTFs,  regions,  and  executives  going  to 
use  these  measures  to  improve  performance  or  will  they  be  discarded. 

Research  Objective 

The  research  objective  of  this  study  is  fourfold.  First,  to  conduct  comparisons 
among  the  different  levels  of  authority  (MTF,  TRICARE  Region,  and  Executive), 
services  (Army,  Navy,  Air  Force),  and  profession  (physician/dentist,  administrator, 
nurse)  regarding  metric  selection  and  use  to  discern  significant  differences.  Second,  to 
request  feedback  from  the  field  (Regions  and  MTFs)  as  to  additional  metrics,  level  of 
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general  understanding,  and  agreement  with  metric  preparation  and  implementation. 

Third,  to  identify,  by  level  of  command,  those  performance  measures  which  are  most  and 
least  applicable  for  decision-making.  Fourth,  to  determine  if  a  major  gap  exists  between 
the  proposed  standard  metric  set  and  executive  level  private  sector  metric  sets. 

Methodology 

This  study  is  composed  of  two  parts.  The  methodology  includes  information  from 
1)  the  private  sector  to  MHS  comparison  and  2)  the  internal  assessment  of  the  proposed 
standard  metric  set.  Under  the  methodology,  the  private  sector  study  will  be  discussed 
first  followed  by  the  internal  MHS  metric  set  assessment  study. 

Sampling  Design 

Private  Sector 

Anthem,  Humana,  Johns  Hopkins  (JHH),  INOVA,  United  Healthcare  (UHC),  the 
American  Association  of  Health  Plans  (AAHP),  and  the  Health  Care  Advisory  Board 
(HCAB)  performance  measurement  information  was  used  to  compare  to  the  MHS.  Dr. 
Sears,  Executive  Director  TMA,  recommended  Anthem  and  Humana  because  they  are 
responsible  for  providing  healthcare  to  TRICARE  beneficiaries  in  regions  two,  three, 
four,  and  five.  UHC,  a  health  plan  similar  to  TRICARE,  was  selected  due  to  availability 
of  data  and  prior  interaction  with  TMA.  OSD  HA  and  TMA  use  information  from  UHC 
as  a  benchmark  for  selected  elements  of  performance.  Johns  Hopkins  and  INOVA,  two 
health  care  systems  consisting  of  a  number  of  hospitals  and  a  system  headquarters,  were 
also  selected  to  ease  comparison  with  the  MTFs  and  TRICARE  regions.  The  AAHP, 
through  A.M.  Best  Company,  annually  compiles  operational  and  financial  metrics 
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representing  results  within  the  HMO  industry.  Finally,  the  Health  Care  Advisory  Board, 
in  their  CEO  Dashboards  publication,  presented  a  list  of  health  plan  and  health  care 
system  performance  measures.  These  lists  were  prepared  from  a  review  of  over  50  health 
care  organizations  and  represent  the  most  “critical  metrics  surfaced  by  the  literature” 
(Berkow  &  Seth,  2000).  These  two  lists  were  also  selected  to  compare  against  the  MHS 
proposed  set,  since  they  represents  the  results  of  a  comprehensive  study  performed  by  the 
HCAB  and  include  both  health  systems  and  plans. 

MHS 

The  sampling  process  for  the  MHS  was  more  complex  and  consisted  of  various 
steps.  First,  the  population  had  to  be  identified.  To  complete  the  study,  it  was  determined 
that  respondents  from  the  three  services,  the  TRICARE  regions,  and  the  MHS  executive 
leadership  were  required.  In  addition,  since  the  study  required  a  sampling  of  personnel 
from  the  three  main  levels  of  “command”,  the  relevant  population  consisted  of  senior 
officials  at  the  MTFs,  senior  officials  at  the  TRICARE  regions,  and  the  executive 
leadership.  Thus,  the  population  for  selection  was  identified  as  the  command  teams  at  the 
Army,  Navy,  and  Air  Force  MTFs  (tri-service),  the  TRICARE  regions  (which  are  tri¬ 
service),  and  the  MHS  executive  leadership  (tri-service  also). 

The  second  step  was  to  size  the  sample  correctly  by  level  of  command.  For  the 
executive  level,  100%  of  the  population  was  selected  (N=9).  The  executive  level  was 
defined  as  the  three  services’  Surgeons  General,  the  Deputy  Surgeons  General,  the 
Deputy  Executive  Director  of  TMA,  the  Executive  Director  of  TMA,  and  the  Acting 
Assistant  Secretary  of  Defense  for  Health  Affairs.  Since  the  population  was  small  all  of 
them  were  selected  to  participate.  Since  the  TRICARE  regions  and  MTFs  were  more 
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numerous,  66%  (2/3)  of  the  total  population  was  selected  as  the  appropriate  sample  size. 
There  was  no  hard  scientific  method  used  for  this  decision,  it  was  based  primarily  on  time 
and  administrative  considerations,  and  a  desire  to  minimize  intrusion  into  daily 
operations.  As  the  literature  (Cooper  &  Schindler,  1998)  states,  there  are  no  set  formulas 
or  processes  to  determine  the  right  sample  size.  Therefore,  the  researcher  was 
comfortable  with  a  selection  representing  two-thirds  of  the  population.  The  command 
teams  for  both  the  regions  and  the  MTFs  were  tapped.  This  amounted  to  three  officers  per 
region  and  four  officers  per  MTF  (N=231).  The  total  N  with  executive  level,  regions,  and 
MTFs  amounted  to  240  candidates. 

The  third  step  was  to  decide  what  type  of  MTF  was  appropriate  and  to  select  the 
sample.  Since  these  measures  include  outpatient  and  inpatient  areas,  hospitals  with 
inpatient  capability  were  chosen  to  participate.  To  select  the  MTFs,  a  stratified  random 
sampling  technique  incorporating  population  proportion  parameters  was  used.  Thus,  the 
sampling  frame  became  an  MHS  MTF  (hospitals)  list  provided  by  the  Defense  Medical 
Facilities  Office  (Appendix  D).  The  list  is  made  up  of  77  hospitals:  28  Army,  25  Navy 
and  24  Air  force.  To  ensure  equitable  sample  selection  by  service,  the  proportion  of  each 
service  to  the  total  was  calculated.  That  proportion  was  used  to  calculate  the  number  of 
hospitals  from  each  service  for  the  research  sample.  For  example:  28  Army  hospitals 
equate  to  36%  of  the  total,  77  hospitals.  The  total  sample  size  (all  services)  equals  51 
hospitals,  which  is  2/3  of  77  hospitals.  Therefore,  the  Army  sample  size  equates  to  18 
hospitals,  which  is  36%  of  51.  The  command  teams  at  the  MTFs  are  composed  of  the 
Commander,  the  Deputy  Commander  for  Nursing,  the  Deputy  Commander  for  Clinical 
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Services,  and  the  Deputy  Commander  for  Administration.  Two  hundred  and  four 
questionnaires  were  mailed  to  the  MTFs. 

The  fourth  step  was  to  select  the  number  of  regions.  Thirteen  TRICARE  regions 
exist  in  the  MHS.  Two-thirds  of  13  equates  to  nine.  Nine  TRICARE  regions  were 
selected  for  the  study.  The  top  officers  at  these  regions  include  the  Director,  the  Deputy, 
and  the  Operations  Director.  Twenty-seven  questionnaires  were  submitted  to  the  regions. 

The  final  step  was  the  selection  process.  A  slip  of  paper  was  prepared  for  each 
candidate  in  the  population  (MTFs  and  Regions).  The  Defense  Medical  Information 
System  (DMIS)  identification  number  represented  each  MTF.  The  numbers  one  through 
13  corresponded  to  the  TRICARE  regions.  The  MTFs  were  stratified  by  service  into  three 
separate  envelopes  and  the  TRICARE  regions  were  all  placed  into  one  envelope.  Two 
personnel  picked  out  the  required  number  of  slips  from  each  envelope.  An  observer  was 
also  present.  The  slips  were  returned  after  each  pick  to  ensure  equal  chance  of  selection 
(Cooper  &  Schindler,  1998).  The  methods  used  in  the  sampling  process  enhanced  the 
chance  that  there  was  equal  probability  for  selection  within  the  population.  See  appendix 
E  for  selected  sample  of  MTFs  and  Regions. 

Research  Design 

The  general  design  was  a  formal  research  study.  It  is  descriptive  in  nature,  with  an 
ex  post  facto  design  (non-experimental),  utilizing  a  communications  tool  (only  MHS)  to 
collect  the  data.  The  comparisons  between  the  private  sector  and  the  MHS  were 
predominately  qualitative.  The  questionnaire  analysis  within  the  MHS  is  quantitative. 
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Private  Sector 

To  determine  differences  between  the  private  sector  and  the  MHS,  the  researcher 
took  the  raw  data  provided  by  these  organizations  and  organized  them  into  domains  or 
performance  measurement  areas.  This  allowed  for  comparisons  at  the  domain  level  and 
facilitated  specific  measurement  identification.  The  key  to  the  design  was  to  enable  a 
step-wise  approach  to  compare  executive  level  metrics  among  the  selected  samples  and 
the  MHS.  A  gap  analysis  was  conducted  to  identify  gaps  between  performance  areas  and 
domains.  This  analysis  also  identified  similarities  in  performance  measures.  This  method 
or  design  enabled  the  researcher  to  compare  systems,  plans,  and  hospitals  to  the  MHS. 
These  organizations  were  asked  to  provide  a  list  of  their  executive  level  metrics,  the 
strategic  goals  they  support,  the  key  components,  and  if  possible  their  effectiveness  at 
driving  behavior. 

MHS 

To  tap  into  the  MHS,  a  three-part  questionnaire  was  sent  to  each  selected  MTF 
and  region.  A  two-part  questionnaire  was  submitted  to  the  executive  level  officers.  The 
questionnaire  was  addressed  to  the  MTF  command  teams  (the  Commander,  the  Deputy 
Commander  for  Clinical  Services,  the  Deputy  Commander  for  Nursing  Services,  and  the 
Deputy  Commander  for  Administration).  At  the  regional  level  it  went  to  the  Director,  the 
Deputy  Director,  and  the  Director  for  Operations. 

Validity  and  Reliability. 

A  panel  of  members  at  the  TMA  determined  content  validity  (how  well  the 
questions  cover  the  topic).  They  reviewed  both  the  general  metric  questions  (part  I)  and 
the  performance  measure  assessment  section  (part  II).  They  ensured  questions  were 
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worded  simply,  relaying  one  concept  or  idea,  avoiding  jargon,  slang  and  use  of  negatives, 
and  portraying  proper  metric  identification  (Spector,  1992).  The  researcher  coordinated 
with  the  TMA  metrics  point  of  contact,  LTC  Dorothy  Smith,  to  formalize  performance 
measurement  inclusion  and  designation  within  the  proposed  standard  metric  set. 

There  was  differing  opinion  at  Health  Affairs  and  TMA  in  conducting  a 
questionnaire  item  analysis.  The  purpose  of  an  item  analysis  “is  to  find  those  items  that 
form  an  internally  consistent  scale  and  eliminate  those  items  that  do  not.”  (Spector, 

1992).  Internal  consistency  is  defined  as  “the  extent  to  which  the  individual  items  that 
constitute  a  test  correlate  with  one  another  or  with  the  test  total”  (Hatcher,  1994). 
According  to  Spector,  the  lack  of  internal  consistency  makes  a  summated  rating  scale 
instrument  somewhat  questionable,  Cronbach’s  Alpha  is  one  of  the  most  utilized 
measures  of  internal  consistency  in  the  social  sciences  (Hatcher,  1994). 

Expert  personnel  at  TMA  reviewed  the  questionnaire  to  assess  if  a  Cronbach’s 
Alpha  test  should  be  administered  to  measure  internal  consistency.  Only  part  1,  made  up 
of  13  questions  would  be  eligible  for  testing.  Part  II,  since  it  only  consisted  of  one 
question  and  a  list  of  47  performance  measures  was  not.  Based  on  the  process  involved  in 
conducting  the  item  analysis,  the  lack  of  a  representative  sample  to  conduct  it  prior  to 
mailing  the  questionnaire,  and  time  constraints,  TMA  recommended  forgoing  a 
Cronbach’s  A.  In  addition,  Spector  (1992)  states  “intercorrelation  can  be  raised  by 
increasing  the  number  of  items  or  by  raising  their  intercorrelations.”  In  an  effort  to  make 
the  questionnaires  simple  and  time  efficient,  the  researcher  kept  the  number  of  items  to  a 
minimum  (13  each).  Time  constraints  would  also  prevent  the  researcher  from  submitting 
various  iterations  of  the  questionnaire  to  improve  item  intercorrelation.  The  researcher 
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forwarded  the  information  to  his  reader  for  approval  to  omit  a  test  for  internal  consistency 
for  part  I  of  the  questionnaire.  Approval  was  granted. 

On  the  other  hand,  Dr.  Bob  Opsut,  Health  Budget  and  Financial  Policy,  Health 
Affairs,  recommended  the  researcher  conduct  a  test  to  measure  consistency  among  the 
scores  of  the  observed  variables.  Though  all  the  desired  steps  (Spector,  1992)  to  conduct 
a  comprehensive  reliability  test  were  not  taken,  a  test  for  internal  consistency  would  show 
evidence  of  intercorrelation.  Dr.  Opsut  said  the  test  would  provide  credibility  to  the 
summated  rating  scale  instrument.  Readers  concerned  with  psychometric  properties 
would  want  empirical  evidence  to  verify  questionnaire  reliability  (Hatcher,  1994). 
Therefore,  scale  reliability  (internal  consistency)  was  assessed  using  coefficient  alpha. 
The  test  was  performed  in  SPSS  using  the  reliability  analysis,  scale  alpha.  Part  1  of  the 
questionnaire  (13  questions)  was  the  only  section  subject  to  testing. 

Table  1.  Mean,  Standard  Deviation,  and  Coefficient  Alpha  Reliability  Results 


Variables 

Mean 

Standard  Deviation 

Coefficient  Alpha 

Std  Item  Alpha 

Items 

5.2 

0.6 

.6933 

.7062 

1  - 13 

A  reliability  coefficient  result  of  .7  or  greater  signifies  an  acceptable  measure  of 
internal  consistency  in  a  summated  rating  scale  instrument  (Nunnaly,  1978).  Though 
modifications  to  increase  item  intercorrelations  were  not  possible,  the  results  were 
encouraging.  Further  refinement  would  increase  internal  consistency;  yet,  for  this  study, 
based  on  time  and  personnel  constraints,  the  initial  questionnaire  instrument  was  used  to 
complete  the  analyses. 
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Crafting  the  Questionnaires 

One  questionnaire  (MTFs  and  Regions)  was  prepared  in  three  parts.  Part  I 
contained  13  statements  concentrating  on  general  aspects  of  strategic  planning  and  metric 
set  preparation.  Part  II  contained  one  question  made  up  of  47  components,  the 
performance  measures  in  the  proposed  standard  set.  To  prevent  any  perception  of 
grouping  or  ranking,  the  performance  measures  were  displayed  in  alphabetical  order.  Part 
III  contained  one  question  requesting  input  from  the  respondents.  See  appendix  H.  The 
second  questionnaire,  for  the  executive  members  of  the  MHS,  did  not  include  part  I 
described  above,  only  parts  II  and  III.  See  appendix  I.  Each  questionnaire  included  a 
section  dedicated  to  demographic  data.  The  respondents  were  asked  to  identify  their 
service,  profession,  rank,  and  job  title. 

Data  Collection 

Private  Sector 

The  researcher  called  the  civilian  healthcare  and  research  organizations  to  request 
performance  measurement  information.  All  civilian  organizations  selected  participated. 
The  quality  and  quantity  of  the  information  varied  noticeably.  Some  organizations  sent 
many  pages  of  information,  while  others  just  one  page.  Numerous  phone  calls  were  made 
to  acquire  the  most  useful  and  pertinent  data.  Data  was  provided  in  various  forms  and  the 
researcher  signed  four  non-disclosure  statements  (Anthem,  Humana,  INOVA,  and  JHH). 
None  of  the  information  provided  was  in  electronic  format.  Face  to  face  meetings  were 
arranged  at  both  JHH  and  INOVA.  The  researcher  was  careful  not  to  bias  data  collection 
from  JHH  or  INOVA  based  on  meetings  or  their  convenient  location.  The  meetings  were 


Metrics 


30 


set  up  to  discuss  basic  strategic  plan  formulation,  structure,  implementation,  and 
performance  measurement  creation. 

MHS 

The  three-part  questionnaire  was  mailed  to  all  MTF  and  TRICARE  region 
respondents  (Appendix  H).  Both  parts  I  and  II  contained  a  number  of  Likert  scale  type 
questions,  thirteen  questions  on  the  first  part  and  one,  with  47  elements,  on  the  second 
part.  Under  part  II,  the  respondents  rated  each  performance  measure  (47  total)  as  to  its 
applicability  in  decision-making  at  their  level  of  command.  Each  question  was  composed 
of  seven  options  from  fully  disagree  to  fully  agree.  Seven  was  selected  over  five  to  fully 
identify  those  just  noticeable  differences  (Dr.  Kenn  Finstuen,  class  lecture,  1999).  A  two- 
part  questionnaire,  omitting  part  I,  was  presented  with  a  three  to  five  minute  brief  to  the 
executive  leadership  (Appendix  I).  The  briefing  was  prepared  to  reflect  the  Questionnaire 
Information  Sheet. 

Mailing  Process 

Prior  to  mailing  the  questionnaires,  the  researcher  requested  up  to  date  addresses 
for  selected  facilities  from  the  service  Surgeons’  General  offices.  Those  offices  verified 
the  existence  of  the  respective  facilities  and  provided  the  latest  mailing  addresses.  Names 
of  commanders  or  command  teams  were  not  requested  due  to  confidentiality  concerns. 

One  envelope  per  facility  (addressed  to  “Commander”)  was  mailed  containing 
four  questionnaires  (MTFs),  three  questionnaires  (regions),  a  cover  letter  addressed  to  the 
commander  or  director  (Appendix  F),  a  questionnaire  information  sheet  (Appendix  G), 
and  addressed  return  envelopes  for  each  questionnaire.  The  questionnaire  information 
paper  included  background  information,  objectives  of  the  study,  methods,  including 
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confidential  and  voluntary  participation,  and  utility  of  the  study.  The  envelopes  were 
mailed  on  04  January  2001  with  a  completion  suspense  date  of  26  January  2001.  The 
researcher  set  06  February  2001  as  the  last  date  to  receive  information.  This  date  was  set 
to  allow  for  delay  in  the  mailing  system.  Based  on  time  and  cost  constraints,  reminder 
cards  were  not  mailed  out.  Self-addressed  business  style  (official  mail)  return  envelopes 
were  included  to  increase  the  likelihood  of  response  (Mangione,  1995) 

The  researcher  had  two  major  concerns  with  the  mailing  process.  The  first  was  the 
decision  to  omit  names  on  the  envelopes.  By  excluding  names,  the  mailing  becomes 
impersonable.  This  might  decrease  response  rates,  and  could  increase  the  likelihood  of 
misdirected  questionnaires. 

The  second  was  the  lack  of  postage  on  the  return  envelopes.  According  to 
Mangione  (1995),  “placing  a  postage  stamp  on  a  return  envelope  puts  subtle  pressure  on 
the  respondent. Administratively,  the  government  post  office  at  TMA  was  not  able  to 
meter  the  envelopes.  The  date  on  the  envelope  must  reflect  the  date  mailed.  In  addition, 
they  were  unable  to  procure  stamps;  therefore,  the  expense  would  be  personal.  The  postal 
personnel  recommended  sending  the  envelopes  without  postage.  Most  military 
organizations  have  metered  postage  capability.  Of  the  120  questionnaires  returned  to 
TMA,  118  were  officially  metered  and  sent  through  government  channels. 

Operational  Definitions  of  Variables 
MBS 

The  dependent  variables  included  the  47  proposed  performance  measures  in  the 
preliminary  standard  metric  set  (p^rt  II  of  the  questionnaire)  and  each  of  the  13  questions 
on  part  I  of  the  questionnaire.  These  variables  were  coded  on  a  Likert  scale,  measured 
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from  one  to  seven.  The  independent  variables  included  type  of  service  (Army,  Navy,  Air 
Force),  profession  (physician/dentist,  nurse,  administrator),  and  level  of  command 
(Executive,  Region,  MTF).  The  Army,  Navy,  Air  Force  were  grouped  under  the  variable 
service  and  coded  1-  Army,  2-  Navy,  3-  Air  Force.  The  same  method  of  coding  was  used 
for  the  other  two  groups.  See  appendix  J  for  a  summary  of  operational  definitions. 

Null  Hypotheses 

Private  Sector 

There  is  no  difference  between  the  domains  and  metrics  within  the  MHS 
preliminary  standard  metric  set  and  selected  private  sector  organizations. 

MHS 

1.  There  is  no  difference  between  services  as  to  metric  selection  and 
implementation  of  a  standard  set. 

2.  There  is  no  difference  between  level  of  command  as  to  metric  selection  and 
implementation  of  a  standard  set. 

3.  There  is  no  difference  between  profession  as  to  metric  selection  and 
implementation  of  a  standard  set. 

Data  Analysis 

Private  Sector 

A  spreadsheet  was  prepared  to  conduct  the  comparison  between  the  MHS  and  the 
private  sector.  All  the  data  was  manually  separated,  categorized,  and  then  compiled  using 
Excel  Microsoft  as  the  application.  A  qualitative  gap  analysis  with  basic  descriptive 
statistical  methods  (frequencies,  means)  was  used  to  discern  similarities  and  differences 
among  the  organizations.  In  addition,  to  determine  magnitude  of  results. 
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proportions/ratios  were  calculated  and  variances  between  the  private  sector  and  the  MHS 
were  analyzed.  Finally,  similar  performance  measures  between  each  civilian  organization 
and  the  MHS  were  identified,  presented,  and  calculated  as  a  percent  of  the  organizations’ 
total.  In  the  prevailing  literature  there  are  no  set  procedures  or  template  for  conducting 
gap  analysis.  A  literature  review  of  gap  analysis  revealed  different  philosophies  of  what  it 
is  and  possible  tools  to  accomplish  one.  The  most  common  application  of  this  method  in 
the  reviewed  literature  was  in  the  customer  service  arena  and  human  service  education 
(Natarajan,  1999,  Dyck,  1996,  Balm,  1996,  Headly  &  Choi,  1992).  The  basic  concept  is 
to  identify  differences  (gaps)  between  the  present  state  and  an  end  state  (Gap  Analysis, 
2000). 

In  this  study  the  present  state  was  defined  as  the  MHS  preliminary  standard  set 
and  the  end  states  were  the  different  sets  used  by  private  sector  healthcare  organizations 
to  include  the  HCAB  and  the  AAHP.  The  variety  of  measures  and  processes  used  in  the 
private  sector  lent  itself  to  a  qualitative  gap  analysis.  The  goal  was  to  identify  general 
differences  and  similarities  in  domains  and  selected  measures.  The  value  to  this  portion 
of  the  study  was  to  identify  the  methods  and  metrics  used  in  the  private  sector.  If  the  gaps 
are  not  excessively  wide,  benchmarking  and  possible  adoption  of  certain  measures  may 
add  value  to  MHS  operations. 

MHS 

The  results  (data)  from  the  questionnaires  were  entered  into  the  Statistical 
Package  for  the  Social  Sciences  software  program  version  10.0  (SPSS).  A  number  of 
questionnaires  contained  incomplete  elements.  The  researcher  considered  filling  those 
gaps  with  the  respective  means.  This  though,  because  the  study  was  comparative  and 
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based  on  analysis  of  variance,  did  not  make  sense.  To  fill  in  the  blanks  with  the  mean 
would  dilute  the  variance,  making  the  data  tighter  around  the  mean.  This  method  would 
create  an  artificial  tightness  of  variance.  In  addition,  according  to  Brenda  Cox,  at 
Mathematica  Policy  Research  in  Washington  DC,  to  fill  in  holes  the  missing  data  should 
be  >  to  10%  of  the  total  results  (Brenda  Cox,  personal  communication,  February  2001). 
This  study’s  questionnaire  only  had  five  items  out  of  60  with  missing  results  equal  to  or 
exceeding  10%.  Furthermore,  LTC  Tom  Williams,  Health  Program  Analysis  and 
Evaluation,  said  that  filling  gaps  for  a  comparison  of  means/univariate  analysis  of 
variance  study  would  artificially  reduce  the  variance  and  modify  the  results  (Tom 
Williams,  personal  communication,  February  2001). 

For  part  I  and  II,  descriptive  statistics,  univariate  analysis  of  variance,  and 
Tukey’s  Honestly  Significant  Difference  test  were  used  to  analyze  the  results.  Means, 
standard  deviations,  and  frequency  distributions  (ungrouped  and  graphed  -  histogram) 
were  used  to  assist  in  determining  distributions  of  item  scores.  Univariate  analysis  of 
variance  is  the  most  elegant  statistical  method  for  this  study.  According  to  Kirk  (1999), 
analysis  of  variance  (ANOVA)  is  the  appropriate  procedure  to  answer  questions 
regarding  differences  among  sets  of  population  means.  If,  after  computing  the  ANOVA, 
there  are  significant  differences,  the  researcher  “may  want  to  draw  some  further 
conclusions”  (Dowdy  &  Wearden,  1991).  A  Tukey’s  Honestly  Significant  Difference 
(HSD)  test  is  one  of  the  most  employed  posteriori  test  to  establish  which  group  means  are 
different  (Dowdy  &  Wearden,  1991 ,  Kirk,  1999).  The  Tukey’s  HSD  test  was  applied  to 
those  population  means  that  were  statistically  significantly  different.  Finally,  exploratory 
factor  analysis  was  conducted  on  part  II  to  identify  any  underlying  constructs  among  the 
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47  performance  measures.  The  items  were  rotated  orthogonally  using  the  varimax 
application  to  simplify  the  structure  and  maximize  the  factor  variances  (Pedhazur  & 
Schmelkin,  1991).  Rotating  factors  allows  for  each  item  to  have  a  consequential  loading 
on  only  one  factor.  This  process  enables  the  researcher  to  ascertain  if  the  selected 
domains  (access,  quality,  cost,  health  outcomes,  and  readiness)  are  actually  indicative  of 
natural  agreement  among  the  respondents. 

The  study  is  comparative;  therefore,  the  tests  conducted  were  appropriate  to 
discern  any  significant  differences  in  opinion  within  and  between  services,  professions, 
or  levels  of  command  in  relation  to  applicability  of  the  proposed  standard  metric  set.  To 
conduct  the  analysis  of  additional  performance  measures  not  included  in  the  proposed  set 
(part  III)  descriptive  statistics  were  used  to  determine  the  top  five  choices  to  incorporate 
in  future  renditions.  Tables  and  graphs  were  prepared  to  present  the  results  in  operational 
and  graphic  form. 

Response  Rates 

Response  rates  were  broken  out  by  level  of  command  and  the  MHS  total.  Of  240 
questionnaires,  120  were  returned  by  06  February  2001.  This  equates  to  a  50%  return 
rate.  Of  the  120  questionnaires  received  at  TMA,  eight  were  discarded  due  to  incomplete 
or  missing  demographic  data.  The  analysis  can  only  be  accomplished  with  complete 
demographic  (profession,  service,  facility  type,  rank)  data.  That  left  the  researcher  with 
112  questionnaires  to  analyze.  The  following  table  reveals  the  number  of  questionnaires 
sent  by  category  and  portrays  the  return  and  completion  rates. 
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Table  2.  Response  and  Complete  Questionnaire  Rates 


Level  of  Command 

Total  Sent 

Returned 

Rate 

Completed 

Rate 

MTF 

204 

96 

47% 

90 

44% 

Region 

27 

19 

70% 

17 

63% 

Executive  Officers 

09 

05 

56% 

05 

56% 

Total  MHS 

240 

120 

50% 

112 

47% 

The  distribution  of  respondents  within  the  independent  variables  (service, 

profession,  level  of  command/authority)  is  presented  in  table  3. 

Table  3.  Independent  Variables 

Components 

Frequency 

Percent  of  Total  (11 

Army 

34 

30.4 

Navy 

31 

27.7 

Air  Force 

47 

42.0 

Physician 

40 

35.7 

Nurse 

29 

25.9 

Administrator 

43 

38.4 

MTF 

90 

80.4 

Region 

17 

15.2 

Executive 

05 

4.5 

Ethical  Considerations 

Ethical  matters  concerning  the  safeguarding  of  participants,  organizations,  subject 
matter,  and  the  researcher  were  covered.  Participation  was  voluntary.  In  the  request  to 
participate,  candidates  were  fully  informed  of  confidentiality  safeguards.  During  the 
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analyses  process,  all  responses  were  held  in  strictest  confidence  and  the  results  were 
shared  in  anonymity  and  in  general  terms.  All  responses  were  coded  and  aggregated  by 
variable  (service,  profession,  and  level  of  command);  thus  no  individual  responses  could 
be  associated  with  specific  units  or  persons.  In  addition,  the  researcher  was  cognizant  of 
his  responsibilities  not  to  exceed  the  scope  of  the  study  and  insure  the  highest  level  of 
accuracy.  Except  for  the  Assistant  Secretary  of  Defense  for  Health  Affairs  and  the 
Executive  Director  of  TMA,  only  active  duty  service  members  participated  in  this  study. 
The  Executive  Director  of  TMA  granted  the  researcher  permission  to  conduct  this  study, 
as  a  tool  to  assess  opinions  from  the  field  and  to  enhance  the  GMP. 

Limitations  of  the  Study 

This  study  has  a  number  of  limitations.  The  first  is  the  selection  of  civilian 
organizations.  The  researcher  selected  one  nationwide  managed  care  plan  (UHC)  that 
closely  resembles  TRICARE.  The  other  two  organizations,  Johns  Hopkins  and  INOVA 
are  regional  organizations  with  excellent  reputations.  The  Executive  Director  of  TMA 
requested  that  the  researcher  review  data  from  Johns  Hopkins.  U.S.  News  and  World 
Report  rates  Johns  Hopkins  as  the  best  hospital  in  the  United  States.  The  Executive 
Director,  TMA  wanted  to  see  what  is  measured  at  the  executive  level  within  Johns 
Hopkins.  Humana  and  Anthem  were  also  selected  at  the  request  of  the  Executive 
Director,  since  the  government  does  business  with  them.  The  research  organizations 
HCAB  and  AAHP  were  selected  due  to  their  significant  studies  in  this  area  of 
management  and  their  available  data.  An  organization  not  selected,  though  considered, 
was  Kaiser  Permanente.  It  is  the  most  similar  to  us,  according  to  LTC  Dott  Smith, 
HPA&E  at  TMA,  but  based  on  time  constraints  the  data  was  not  available. 
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A  second  limitation  is  the  selection  of  regions  and  MTFs.  Although,  the 
researcher  insured  random  sampling,  there  is  an  opportunity  to  select  non-representative 
MTFs  or  regions  that  would  decrease  the  ability  to  generalize  the  findings.  A  third 
limitation  concerns  the  knowledge  base  of  those  selected  command  teams.  Will  they  be 
familiar  enough  with  the  concepts  to  answer  the  questions  from  a  knowledgeable 
position?  A  fourth  limitation  is  with  the  ability,  experience,  and  knowledge  of  the 
researcher.  For  example,  did  the  researcher  construct  a  questionnaire  that  properly 
identifies  and  covers  all  the  pertinent  issues?  In  this  case,  the  questionnaire  validation 
exercise  by  a  panel  of  experts  at  TMA  mitigates  some  of  the  error.  But  the  limitation 
remains,  due  to  an  inexperienced  researcher. 

A  fifth  limitation  is  the  questionnaire  return  rate  and  accuracy.  With  all  the 
planning  and  operational  demands  faced  by  the  command  teams,  did  they  conscientiously 
fill  out  and  return  the  questionnaire.  Without  making  it  an  official  TMA/SG  document,  it 
will  be  up  to  the  respondent  to  take  this  study  seriously  enough  to  expend  the  required 
effort.  Finally,  the  last  limitation  also  involves  the  command  teams.  How  did  they  view 
the  exercise?  Did  they  put  a  spin  on  their  replies  or  did  they  answer  as  honestly  as 
possible?  The  confidentiality/ethical  measures  implemented  to  protect  the  respondents 
hopefully  ameliorated  this  concern. 

Findings  and  Discussion 

The  findings, and  discussion  portion  of  the  paper  are  portrayed  together  in  five 
parts.  The  five  parts  are  the  general  metric  statements  (part  I,  questionnaire),  the  standard 
metric  set  (part  II,  questionnaire),  the  additional  performance  measures  and  domains 
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recommended  by  the  respondents,  the  identification  and  analysis  of  underlying 
constructs,  and  the  comparison  with  the  private  sector. 

Part  One  of  Questionnaire 

The  results  of  the  first  part  of  the  questionnaire,  consisting  of  general  metric  and 
strategic  planning  statements,  exhibited  one  significant  difference  out  of  13  statements. 
On  one  statement,  the  respondents  disagreed  as  to  its  meaning  or  utility.  The  difference 
was  significant  when  taking  into  account  the  effects  of  profession  and  level  of  command. 
Those  two  independent  variables  combined  display  a  statistical  significant  F  value  at  the 
.017  confidence  level  (Appendix  K). 


Table  4.  Measures  with  a  significant  F  value  at  the  .05  level. 


Measures 

Source 

F  value 

sig. 

Metric  sets  are  currently  tied  (mapped) 
back  to  management  objectives 

Level  of  Command* 
Profession 

4.288 

.017 

A  Tukey’s  HSD  test  discerned  the  significant  difference  to  be  between  the  Air 
Force  and  the  Army  (Appendix  L).  On  the  surface,  this  does  not  fully  support  the  finding 
that  profession  and  level  of  command  were  responsible  for  the  effect.  Unfortunately 
though,  the  post  hoc  test  was  not  complete  due  to  lack  of  data.  Level  of 
command/authority  was  not  included  because  there  were  fewer  than  three  groups.  In 
addition,  there  was  only  one  profession  present  in  the  Navy  at  the  regional  level.  These 
realities  confounded  the  results,  making  them  suspect. 

This  lack  of  variance  skewed  the  results  of  the  Tukey’s  test.  Considering  the 
inability  to  compute  a  post  hoc  test  properly,  a  comparison  of  means  was  used.  The 
difference  between  the  means  of  the  two  levels  (MTF  and  Region)  was  .89  (4.42  -  3.53). 
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This  difference  was  the  second  highest  when  compared  with  the  other  items.  This  may  be 
the  driver  behind  the  results  of  the  Univariate  ANOVA.  Regardless  of  the  descriptive 
statistical  results  (.89),  estimating  the  effects  of  interaction  between  the  levels  of 
authority  was  not  possible.  Thus,  the  researcher  can  only  conclude  that  a  significant 
difference  exists  but  cannot  statistically  display  the  exact  origin  of  the  difference. 

The  means  and  standard  deviations  are  presented  in  the  following  table.  The  table 
is  organized  from  highest  mean  score  to  lowest  mean  score  based  on  the  Likert  scale 
responses.  The  means  range  from  6.03  to  4.29  and  the  standard  deviations  range  from 
1.09  to  1.78  on  a  seven-point  scale.  Higher  item  scores  denote  that  the  respondents 
agreed  that  those  items  were  more  applicable  for  decision-making  at  their  organization 
than  they  did  with  the  lower  scored  items. 

Table  5.  Statements,  Means,  and  Standard  Deviations  for  Part  I,  Questionnaire 


Statements 

Mean 

SD 

A  standardized  methodology  to  calculate  performance 
measures  should  be  instituted  throughout  the  MHS 

6.03 

1.09 

Too  many  metric  sets  exist  in  the  MHS 

6.01 

1.29 

Services  should  review  metric  sets  against 
management  objectives 

6.00 

1.50 

Performance  measures  help  organizations  achieve 
strategic  goals 

5.56 

1.25 

A  MHS  standard  metric  set  should  be  used  as  a  template 
for  other  service,  regional,  or  MTF  sets 

5.47 

1.53 

I  am  aware  of  corporate  management  objectives 
reflected  in  the  MHS  strategic  plan 

5.42 

1.47 

I  am  aware  of  service  medical  specific  strategic  goals 

5.27 

1.77 

Standard  metric  set  should  be  implemented  in  the  MHS 

5.21 

1.45 

Table  Continues 
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Statements 

Mean 

SD 

The  standard  metric  set  supports  enterprise  wide 
strategic  goals 

4.74 

1.10 

The  standard  metric  set  in  part  II  represents  a  balanced 
approach  to  metric  selection. 

4.70 

1.29 

Standard  metric  set  supports  service  specific  strategic  goals 

4.65 

1.23 

Metric  sets  are  best  prepared  under  a  single  proponent 

4.63 

1.78 

Metric  sets  are  currently  tied  (mapped)  back 
to  management  objectives* 

4.29 

1.34 

*  Significant  difference  at  the  P  =  .05  level. 


The  findings  on  this  first  part  denote  that  respondents  are  likely  to  agree  on  most 
of  the  statements.  The  interesting  outcome  is  not  so  much  the  disagreement  exhibited  in 
whether  sets  are  currently  mapped  back  to  management  objectives,  but  the  agreement  and 
relatively  high  scores  on  certain  items.  Not  only  did  some  of  these  items  exhibit  a  high 
score;  the  highest  scored  had  the  lowest  standard  deviation,  denoting  little  disagreement 
among  respondents. 

Of  the  remaining  12  items,  three  are  at  a  score  of  6.00  or  higher.  The  respondents, 
regardless  of  service,  profession,  or  level  of  authority/command  believe  that  those  items 
should  be  of  high  priority  in  metric  set  creation  and  implementation.  The  highest  rated 
item,  which  also  exhibited  the  lowest  standard  deviation,  was  a  desire  to  institute  a 
standardized  methodology  to  calculate  performance  measures.  Of  note  also  is  the  low 
score  and  high  disagreement  (standard  deviation)  for  the  item:  metric  sets  are  best 
prepared  under  a  single  proponent.  This  may  lead  one  to  conclude  that  the  respondents 
are  interested  in  maintaining  their  right  to  create  measures  within  their  organizations.  In 
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addition,  the  proposed  standard  metric  set  was  rated  low  in  its  ability  to  support  or  tie 
back  to  service  specific  goals.  It  was  also  rated  rather  poorly  in  supporting  MHS  wide 
goals. 

The  respondents  rated  both  creating  a  standard  set  to  use  as  a  template  and 
implementing  a  standard  set  as  5.47  and  5.21  with  standard  deviations  of  1.53  and  1.45 
respectively.  They  were  ranked  fifth  (template)  and  eighth  (implement)  out  of  13  on  the 
list  above.  The  standard  deviations  for  the  two  items  are  above  the  mean  standard 
deviation,  1.39.  This  represents  a  higher  level  of  disagreement  among  the  respondents  for 
standard  set  implementation  and  use  as  a  template  than  the  average  level  of  disagreement 
for  all  the  items  reflected  in  part  I  of  the  questionnaire. 

The  value  of  this  part  of  the  questionnaire  turned  out  to  be  the  overall  agreement 
among  the  respondents  with  the  various  statements  regarding  metric  set  design  and 
implementation.  In  addition,  the  respondents  identified  certain  items  of  importance  that 
are  central  to  future  metric  set  implementation.  These  items  actually  validated  some  of 
the  Acting  Assistant  Secretary  of  Defense’s  and  the  DMOC’s  guidance  to  TMA. 
Specifically,  to  identify  all  current  MHS  created  metric  sets  and  streamline,  to  identify 
externally  directed  sets,  to  develop  a  metric  framework,  and  to  standardize  calculation 
methodologies.  The  general  portion  of  the  questionnaire  has  confirmed  the  need  for  a 
comprehensive  review  and  analysis  of  the  enterprise’s  performance  measurement 
process.  The  paper  proceeds  to  reviewing  differences  and  similarities  in  opinion  among 
the  respondents  regarding  the  proposed  standard  metric  set,  part  II  of  the  questionnaire. 
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Part  Two  of  Questionnaire 

Though  the  methodology  used  to  analyze  the  data  in  part  II  is  similar  to  part  I,  the 
numbers  of  data  elements  makes  it  much  more  complex.  Of  the  47  performance  measures 
on  the  proposed  metric  set,  six  measures  revealed  a  significant  F  value  at  the  p  =  .05 
confidence  level  (Appendices  M  through  R). 

Table  6.  Measures  with  a  significant  F  value  at  the  .05  level. 


Measures 

Source 

F  value 

sig. 

%  AD  members  with  temporary 
medical  profiles  (Readiness) 

Service*Profession 

2.786 

.031 

Relative  weighted  product 
per  1000  enrollees  (Cost) 

Profession 

5.551 

.006 

Managed  Care  Support  Contractor 
phones  ,  %  of  all  lines  busy  (Access) 

Service*LvlCmd 

2.492 

.049 

Malpractice  claims  filed  per 

100  physicians  (Quality) 

Service*Profession 

2.943 

.024 

Dental  Wellness  (%  class  1) 

Serv*LvlCmd*Prof 

6.535 

.012 

(Readiness) 

Service*Profession 

2.909 

.026 

Dental  Readiness  (%  class  1  &2) 

Serv*LvlCmd*Prof 

5.169 

.025 

(Readiness) 

Service*Profession 

2.733 

.034 

Three  of  these  measures  fall  under  the  readiness  domain,  one  under  quality,  one  under 
access,  and  one  under  cost.  It  is  interesting  to  note  that  of  the  four  measures  in  the 
readiness  performance  measurement  area,  three  show  a  significant  F  value,  75%  of  the 
total.  This  result  may  indicate  that  the  respondents  do  not  have  agreement  as  to  common 
measures  of  military  medical  readiness.  The  researcher  initially  observed  that  in  many 
meetings  to  discuss  applicability  of  measures,  the  greatest  disagreement  among  services 
and  attendees  tended  to  center  around  what  readiness  measures  to  include.  The  different 
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service  missions  and  cultures  relating  to  wartime  activities  shaped  the  results  within  the 
readiness  performance  measurement  area. 

To  discriminate  just  exactly  where  those  differences  lie,  the  following  tables 
break  down  the  three  groups  (service,  profession,  and  level  of  command)  by  performance 
measure.  The  tables  are  based  on  the  Tukey’s  HSD  output  (Appendices  S  through  X). 
Table  7.  Profiles* 


Service 

Profession 

Level  of  Command 

N/A 

N/A 

N/A 

*Due  to  gaps  in 
HSD  test. 

within  groups,  significant  differences  were  not  identified  in  the  Tukey’ 

Table  8.  Relative  Weighted  Products  per  Enrollee 

Service 

Profession 

Level  of  Command 

N/A 

Nurse  and  Admin 

Sig.  .009 

N/A 

Table  9.  MCSC  Phones  %  busy* 

Service 

Profession 

Level  of  Command 

N/A 

N/A 

N/A 

*Due  to  gaps  in  within  groups  and  F  value  at  the  .049  significance  level,  significant 
differences  were  not  identified  in  the  Tukey’s  HSD  test. 


Table  10.  Malpractice  Claims  Filed 

Service 

Profession 

Level  of  Command 

N/A 

Nurse  and  Admin 

N/A 

Sig.  .004 
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Table  11.  Dental  Wellness 


Service 

Profession 

Level  of  Command 

N/A 

Nurse  and  Admin 

MTF  and  Region 

Sig.  .006 

Sig.  013 

Table  12.  Dental  Readiness 

Service 

Profession 

Level  of  Command 

Army  and  Air  Force  Physicians  and  Admin  MTF  and  Region 

Sig.  .025  Sig.  .038  Sig.  .001 


Region  and  Exec  Ldrs 
Sig.  .040 

Unfortunately,  it  is  obvious  that  in  two  measures  post  hoc  test  results  are  not 
significant.  After  deliberation  with  Dr.  Mangelsdorff,  Baylor  University,  it  was 
determined  that  the  results  could  be  skewed  due  to  missing  types  of  professions.  This  is 
similar  to  the  phenomena  faced  with  the  Tukey’s  HSD  results  in  part  I,  where  the 
researcher  can  identify  a  significant  effect  among  the  variables,  but  cannot  pin  point  the 
source  of  such  effect. 

For  the  two  items  without  significant  results  on  the  Tukey’s  HSD,  the  researcher 
analyzed  the  descriptive  statistical  results  where  the  F  value  was  significant.  The  means 
for  profiles  illustrate  that  the  Air  Force  felt  it  was  most  applicable  (4.63)  and  the  Navy 
the  least  (4.00).  Reviewing  professions,  the  nurses  considered  profiles  most  applicable 
(4.89)  and  the  administrators  the  least  applicable  (4.17)  for  decision-making  purposes  and 
inclusion  in  the  standard  set.  That  is  an  interesting  outcome,  since  one  would  assume  that 
administrators  are  more  sensitive  to  temporary  profiles  since  it  affects  readiness  and 
deployability  of  the  force.  For  the  second  item,  the  means  for  Managed  Care  Support 
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Contractor  (MCSC)  phone  lines  %  busy,  the  level  of  command  variable  indicated  the 
greatest  standard  deviation.  The  TRICARE  region  respondents  considered  it  most 
applicable  (5.09)  and  the  executive  level  least  applicable  (3.50)  for  inclusion  into  the 
proposed  standard  set.  This  makes  sense,  since  at  the  regional  level;  the  leadership  would 
be  interested  in  contractor  response  to  the  served  beneficiary  population.  This  score 
indicates  that  this  measure  loses  some  of  its  value  at  the  executive  level. 

The  prominent  results  as  noted  in  Tukey’s  HSD  are  in  the  dental  wellness  and 
readiness  measures.  Major  disagreements  as  to  the  utility  of  these  measures  in  the 
decision  making  process  was  evident  among  professions  and  levels  of  command.  The 
nurses  ranked  dental  wellness  5.34  and  the  administrators  4.56.  This  may  indicate  that 
nurses  are  more  attune  to  wellness  measures.  The  hospital  command  teams  considered 
this  measure  important  to  track  (5.68),  while  the  regions  and  the  executive  leaders  rated  it 
at  5.30  and  4.56  respectively.  This  result  may  point  to  a  focus  on  wellness  at  the  point  of 
medical  care  delivery. 

The  Army,  Navy  and  Air  Force  rated  dental  readiness  at  5.09,  5.33,  and  5.89 
respectively.  It  is  interesting  to  note  that  based  on  a  briefing  presented  at  DoD,  the 
number  of  military  members  deployed  is  greatest  in  the  Air  Force,  then  the  Navy,  and 
finally  the  Army.  As  far  as  profession  is  concerned,  the  physicians  considered  this  most 
applicable  at  5.80,  the  nurses  next  at  5.71,  and  the  administrators  at  5.07.  The  executive 
level  of  command  rated  this  variable  at  6.00,  the  MTFs  at  5.70,  while  the  TRICARE 
Region  command  team  rated  it  at  4.35.  This  difference  pinpoints  the  wide  gap  between 
the  focus  at  the  region  (health  plan)  and  the  executive  level  in  the  area  of  readiness.  At 
the  TRICARE  region,  the  focus  is  on  the  delivery  and  management  of  the  peacetime 
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healthcare  product.  Therefore,  readiness  concerns  are  not  at  the  forefront.  Readiness 
becomes  more  important  at  both  the  MTF  level  and  the  executive  level.  The  MTFs  are 
the  focal  point  for  care  and  have  assigned  personnel  targeted  to  support  deploying  units 
(platforms).  At  the  executive  level  those  leaders  are  concerned  with  all  aspects  of 
healthcare,  not  specifically  the  provisions  of  peacetime  care. 

Though  the  differences  are  interesting,  what  is  more  remarkable  is  the  similarity 
among  all  the  respondents.  Of  the  47  performance  measures  under  review,  41  did  not 
reveal  any  significant  difference  when  rated  for  applicability  in  the  decision-making 
process  among  services,  levels  of  command/authority,  and  profession.  This  was 
surprising,  because  it  affirms  that  the  working  group  has  devised  a  comprehensive 
instrument  for  possible  use  as  a  performance  measurement  template.  In  part  I  of  the 
questionnaire,  the  respondents  rated  the  statement  regarding  preparation  of  metric  sets  by 
a  single  proponent  next  to  last.  Yet,  under  a  general  assessment  based  on  levels  of 
disagreement  between  respondents,  it  is  evident  that  a  single  proponent  can  adequately 
prepare  a  comprehensive  tool  for  use  in  the  field  and  at  the  executive  level.  This 
disagreement  in  results,  between  part  I  and  part  II,  point  to  a  concern  with  loss  of 
autonomy.  Though  this  proposed  set  may  adequately  represent  the  decision-making  needs 
of  the  respondents,  they  do  not  want  to  lose  their  right  to  create  additional  measures  to 
track  organizational  priorities  not  reflected  on  the  standard  set.  Respondent  agreement, 
with  few  exceptions,  denotes  that  they  concur  with  performance  measurement  selection 
for  a  MHS  wide  proposed  standard  set. 
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More  surprisingly  though,  is  a  ranking  based  on  the  means  of  the  performance 
measures.  The  following  table  lists  all  rated  47  measures  from  highest  to  lowest.  The 


measures  displaying  significant  differences  in  opinion  are  italicized. 
Table  13.  Performance  Measures  in  Order  of  Importance 


Performance  Measure 

Means 

Standard 

Deviations 

Satisfaction  with  Quality  of  Health  Care 

6.16 

1.11 

%  Beneficiary  Satisfied  with  Customer  Service 

6.15 

1.07 

Patient  Satisfaction  with  Access 

6.13 

1.04 

Satisfaction  with  Access  to  Healthcare 

6.09 

1.11 

Overall  Satisfaction  with  Clinic  Visit 

6.07 

1.04 

Patient  Satisfaction  with  Waiting  Times 

5.94 

1.16 

%  Eligible  Enrolled  to  MTF 

5.91 

1.25 

Cost  per  Enrollee 

5.89 

1.15 

Ease  of  Making  Appointment  by  Phone 

5.89 

1.23 

Emergency  Room  Visits  per  1000  Enrollees 

5.87 

1.33 

Preventive  Care  Measures  (ex.  Pap  smears) 

5.81 

1.18 

Pharmacy  Cost  per  Member  per  Month 

5.78 

1.35 

Satisfaction  with  Access  to  Specialist 

5.70 

1.19 

Direct  Care  Market  Share  -  Outpatient  workload 

5.69 

1.40 

Wait  Time  for  Appointment  (TRICARE  Stds) 

5.62 

1.34 

%  of  Enrollees  Seen  by  their  PCM 

5.58 

1.36 

Wait  Time  at  Appointment  (30  minute  standard) 

5.54 

1.15 

Visits  per  Provider  per  Day 

5.53 

1.24 

Cost  per  Visit  Weighted 

5.52 

1.39 

Dental  Readiness  (%  Class  1  and  2)* 

5.50 

1.46 

%  Providers  Medical  Readiness  Tmd  &  Certified 

5.41 

1.59 

Cost  per  Bed  day 

5.39 

1.38 

Preventable  Admission  Rates  for  Prime  Enrollees 

5.35 

1.45 

Table  Continues 
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Performance  Measures 

Means 

Standard 

Deviations 

Cost  per  RWP  (Relative  Weighted  Product) 

5.34 

1.47 

Claims  Processed  within  30  Days 

5.32 

1.43 

Visits  per  1000  Weighted 

5.15 

1.34 

Dental  Wellness  (%  Classl)* 

5.12 

1.59 

Direct  Care  Market  Share  -  Inpatient  workload 

5.08 

1.66 

MCSC  %  of  Calls  Answered  within  2  minutes 

5.05 

1.41 

Laboratory  Costs  per  Member/Month 

5.02 

1.59 

Radiology  Costs  per  Member/Month 

4.96 

1.62 

SADR  Completion  Rate 

4.93 

1.53 

SDDR  Completion  Rate 

4.91 

1.56 

Capital  Costs  per  Member  per  Month 

4.88 

1.74 

Bed  days  per  1000  Enrol  lees 

4.86 

1.68 

MCSC  Phones  —  %  of  all  lines  busy* 

4.83 

1.50 

Dispositions  per  1000  Enrollees 

4.78 

1.53 

Nurse  FTE  per  Weighted  Bed  Day 

4.74 

1.57 

Beneficiary  Grievances/ 1000  Enrollees 

4.70 

1.67 

RWPs per  1000  enrollees* 

4.68 

1.59 

Health  Status  (self  reported  on  SF  12 — mean) 

4.64 

1.49 

MEPRS  Completion  Rate 

4.41 

1.84 

%  of  AD  Members  with  Temp  Medical  Profiles* 

4.38 

1.74 

Beneficiary  Appeals  per  1000  Claims 

4.21 

1.64 

JCAHO  Grid  Scores 

4.19 

1.76 

Malpractice  Claims  Paid  per  100  Physicians 

4.09 

1.78 

Malpractice  Claims  Filed  per  100  Physicians* 

4.05 

1.79 

*Significant  at  the  .05  level. 


In  addition  to  the  table,  frequencies  for  each  measure,  per  rating  (1  through  7),  are 
displayed  in  graphical  form  (histograms)  in  appendix  Y  (Y1  through  Y47). 
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A  couple  observations  are  instantly  evident.  The  top  six  measures  are  satisfaction 
based.  Three  are  from  the  quality  domain  and  three  from  the  access  domain.  Of  the  top 
ten  measures  only  one  measure  is  concerned  with  cost,  cost  per  enrollee.  Of  the  top  20 
measures  with  a  mean  of  5.50  or  higher,  four  are  cost,  one  is  readiness,  one  is  health 
outcome,  four  are  quality,  and  nine  are  access  measures.  If  one  decomposes  the  domains 
and  reviews  these  measures  from  a  customer-oriented  perspective,  12  of  the  top  20 
measures  are  specifically  related  to  customer  satisfaction. 

These  12  patient  oriented  measures  also  exhibit  a  low  average  standard  deviation, 
1.16.  As  compared  to  the  other  measures,  not  specifically  relating  to  customer 
satisfaction,  this  1.16  standard  deviation  is  the  lowest.  This  represents  responses  that  are 
tight  around  the  mean,  with  evident  agreement  among  the  respondents  in  rating  customer 
satisfaction  measures  at  the  top.  Seventy-five  percent  of  the  respondents  rated  the  top  five 
measures  as  a  6.00  or  higher.  These  measures  are  all  directly  related  to  customer 
satisfaction. 

In  the  top  25%,  a  total  of  12  measures,  one  health  outcome  measure  is  present. 
This  preventive  care  measure  was  rated  number  ninth.  The  other  health  care  outcome 
measure,  self-reported  health  status,  was  rated  41.  This  result  leads  one  to  believe  that 
self-reported  measures  are  not  as  reliable  as  clinically  derived  health  outcome  tests.  In 
this  vein,  the  respondents  do  not  consider  this  measure  as  critical  to  decision  making  as 
others  based  on  its  subjective  nature. 

The  mean  score  of  all  the  means  is  5.25  (grand  mean).  Twenty-five  measures  are 
distributed  above  the  grand  mean  and  22  below.  Assuming  a  normal  distribution,  68%  of 
the  measures  will  fall  within  one  standard  deviation  above  and  below  the  mean. 
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Simplistically,  one  standard  deviation  encompasses  32  measures.  On  the  low  end  of  the 
rankings,  six  measures  are  outside  one  standard  deviation.  These  six  lowest  ranked 
measures  have  mean  scores  ranging  from  4.41  to  4.05,  with  standard  deviations  ranging 
from  1.79  to  1.64.  The  measures  are  distributed  under  the  following  domains,  four  in 
quality,  one  in  readiness,  and  one  in  special  interest.  The  standard  deviations  with  these 
low  ranking  measures  are  wider  than  with  the  high-ranking  measures,  revealing  higher 
disagreement  among  respondents  as  to  their  ranking  within  the  standard  set.  The  quality 
measures  are  based  on  malpractice  claims  filed,  malpractice  claims  paid,  beneficiary 
appeals,  and  JCAHO  grid  scores.  These  measures  are  more  applicable  at  the  health  plan 
or  regional  level  than  at  the  hospital.  According  to  Berkow  and  Seth  (2000),  these  were 
measures  selected  for  inclusion  with  health  plan  vice  hospital  dashboards.  The  majority 
of  the  cost  measures  fell  within  one  standard  deviation  of  the  grand  mean.  To  reduce  the 
number  of  measures  within  the  set  to  approximately  40,  as  recommended  by  the  Health 
Care  Advisory  Board  (Berkow  &  Seth,  Health  Plan  Measures,  2000),  based  on  the  results 
of  this  questionnaire,  the  MHS  would  lose  40%  of  its  quality  based  measures.  These  are 
the  last  ranked  measures  in  the  proposed  set. 

Interpreting  these  results  leads  one  to  conclude  that  command  teams  at  the  various 
command  levels  and  services  agree  that  customer  satisfaction  measures  are  most 
applicable  for  use  within  the  enterprise.  The  focus  in  MHS  facilities  is  to  improve  the 
interaction  between  the  provider  of  care,  to  include  administrative  functions,  and  the 
customer.  This  may  be  a  factor  of  a  desire  to  improve  customer  satisfaction  with  services 
provided,  increase  the  enrolled  population  and  market  share,  decrease  beneficiary 
reliance  upon  the  managed  care  contractors,  and  increase  productivity  at  the  military 
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facilities.  This  coincides  with  TRICARE’ s  latest  strategic  goal,  to  improve  patient 
satisfaction  (Cowan,  2001).  The  field  is  in  synch  with  TMA’s  focus  to  improve  customer 
satisfaction.  This  synergy  is  a  result  of  effective  communication  and  cooperation  between 
TMA  and  the  services.  A  number  of  these  satisfaction  measures  are  leading  indicators 
that  enable  leaders  to  track  performance,  modify  operations,  and  change  employee 
behavior  to  increase  productivity,  market  share,  and  ultimately  decrease  dependence  upon 
the  purchased  care  system. 

The  respondents  also  agree  that  cost  measures  are  of  medium  importance  since 
they  all  clustered  within  one  standard  deviation  from  the  grand  mean  score,  with  an 
average  standard  deviation  of  1.47.  Considering  the  high  priority  placed  on  controlling 
cost  at  the  highest  levels  in  DoD,  it  is  eye  opening  to  see  that  these  measures  do  not  rate 
higher  on  the  applicability  scale.  Though  ascertaining  causes  for  their  ranking 
distributions  is  inappropriate,  a  number  of  reasons  for  the  results  can  be  discussed. 

For  example,  some  measures  are  similar  (cost  per  visit  weighted  and  visits  per 
1000  weighted,  bed  days  per  1000  and  dispositions  per  1000  and  RWPs  per  1000).  Most 
of  the  cost  measures  are  lagging  indicators;  they  describe  what  occurred  but  do  not 
answer  the  question  why  it  occurred  or  assist  in  predicting  future  position.  Further,  many 
command  teams  may  feel  that  they  are  not  in  control  of  their  budget.  Decisions  made  at 
their  level  will  have  little  impact  on  overall  future  funding  allocations.  This  is, 
unfortunately,  one  of  many  shortcomings  of  the  federal  fiscal  process.  Next,  the 
purchased  care  budget  is  carved  out  and  depending  on  the  region,  it  is  under  revised 
financing  or  under  TRICARE  1.0.  This  difference  forces  healthcare  leaders,  managers, 
and  providers  to  run  two  different  TRICARE  systems  encompassing  a  number  of  diverse 
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business  rules.  This  contractual  system  is  highly  complex.  The  majority  of  those  MTF 
administrators  do  not  understand  the  intricacies  of  the  Bid  Price  Adjustments  (BPA), 
resource  sharing  agreements,  and  effects  of  workload  variances.  Finally,  though  the  list 
can  go  on,  what  is  the  incentive?  Midway  through  the  year,  the  system  will  get  a 
Congressional  supplemental  or  the  DoD  will  reprogram.  In  this  vein,  commanders  are  not 
encouraged  to  create  efficiencies  because  historical  funding  processes  actually  reduce 
funding  allocations  regardless  of  requirements.  Therefore,  financial  measures,  though 
critical  at  the  highest  levels  within  DoD,  may  prove  not  to  be  the  focus  at  the  operational 
level. 

The  quality  measures  that  were  rated  lowest  are  measures  relating  to  MHS 
concerns.  These  measures  do  not  echo  the  hard  clinical  quality  or  health  plan  measures 
encountered  at  hospitals  and  other  healthcare  facilities.  These  clinical/health  plan  quality 
measures  were  absent  from  the  proposed  standard  set.  Hard  clinical  quality  measures 
reviewed  at  hospitals  and  health  plans  include  unplanned  return  to  specialty  care  and 
operating  room  rates,  patient  fall  with  injury  rate,  number  of  serious  adverse  drug  events, 
and  acute  care  readmission  rate,  to  name  a  few.  These  and  similar  quality  measures  may 
have  received  higher  ratings,  at  least  with  the  MTF  command  teams.  It  would  be 
beneficial  to  study  the  effectiveness  of  tracking  some  of  the  lowest  ranked  measures  to 
possibly  delete  them  from  the  set.  If  all  parties  agree  that  they  are  unimportant,  analyzing 
those  measures’  results  have  little  impact  on  improving  operations. 

A  quick  review  of  the  six  lowest  ranked  measures,  outside  one  standard  deviation, 
showed  that  none  of  the  six  had  a  mean  rating  among  the  independent  variables  (service, 
level  of  command,  profession)  of  5.01  or  greater.  Most  of  the  ratings  fell  around  4.10.  For 
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example,  JCAHO  grid  score  results,  a  measure  of  clinical  quality,  was  rated  lowest 
among  physicians  (3.68).  This  may  denote  that  providers  do  not  consider  this  measure  to 
be  of  value.  Improving  clinical  quality  and  modifying  provider  behavior  may  be  better 
served  with  other  more  accepted  and  objective  measures  of  performance.  Another 
example  is  malpractice  claims  paid.  According  to  TMA  this  is  a  measure  of  clinical 
quality,  yet  the  administrators  in  the  field  gave  it  a  3.45  rating.  It  could  be  argued  that  this 
might  not  be  a  clinical  quality  rating  but  actually  a  rating  depicting  the  validity  of  claims 
filed.  If  so,  what  is  the  objective  of  this  performance  measure,  is  it  clinical  quality  or 
claims  administration/legal  effectiveness.  Measures  such  as  this  one  that  can  be  used  in 
various  ways  to  meet  a  number  of  objectives  could  generate  unintended  employee 
behaviors  and  perverse  incentives.  Reviewing  the  purpose  of  each  metric  to  ensure  it 
measures  processes  and  outcomes  in  accordance  with  selected  goals  is  important  to 
decreasing  the  possibility  of  perverse  incentives  and  missed  management  objectives. 

Part  Three  of  Questionnaire 

Since  some  measures  were  rated  higher  than  others  as  far  as  applicability  for 
decision  making  in  the  respondents’  respective  service,  level  of  command,  and 
profession,  some  measures  may  exist  that  were  not  incorporated  in  the  proposed  standard 
set.  Possibly,  the  work  group  did  not  include  all  the  pertinent  metrics  used  by  services 
and  professions  within  specific  levels  of  command.  Therefore,  it  is  beneficial  to  analyze 
any  additional  metrics  included  in  the  questionnaires  for  broad  areas  of  agreement  and 
frequency.  Part  III  of  the  questionnaire  asked  the  respondents  to  add  any  performance 
measures,  that  in  their  estimation  where  missing  from  the  proposed  standard  set.  The 
results  are  presented  by  service  in  appendix  Z  and  by  level  of  command  in  appendix  AA. 
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The  researcher  grouped  the  recommended  measures  into  17  separate  domains  to 
identify  specific  differences  in  opinion.  The  domains  created  are  Utilization,  Employee 
Satisfaction,  Stakeholder  Satisfaction,  Staffing,  PMPM  Cost,  Quality,  Health  Status, 
Preventive  Care,  General  Cost,  Direct  Care  vs.  Purchased  Care,  HEDIS,  Resource 
Availability,  Enrollment,  Billing/Claims,  Patient  Acuity/Case  Mix  Index,  Readiness,  and 
Provider  Certification.  The  most  popular  domains  were  utilization  (12  hits),  staffing  (9 
hits),  and  Employee  Satisfaction  (6  hits).  Hits  are  defined  as  the  number  of  times  a 
measure  was  included  as  an  addition  in  part  III.  For  example,  the  respondents  did  not  add 
12  new  utilization  measures,  but  either  stated  more  utilization  rates  or  gave  examples  of 
specific  measures.  Some  of  these  specific  measures  were  repeated  among  different 
respondents.  The  five  most  requested  additions  (ranked)  are  1)  percent  providers  satisfied 
with  job  and  TRICARE,  2)  percent  staffing  of  PCM  support  staff/positions,  3)  visits  per 
provider  per  day,  weighted,  4)  percent  time  providers  at  clinic,  administration,  or  other 
duties,  5)  number  of  visits  per  member  per  month  (appendix  AB). 

The  three  most  popular  domains,  utilization,  staffing,  and  employee  satisfaction 
exhibited  some  interesting  results.  Based  on  the  distribution,  under  utilization  the  Army, 
Navy,  and  Air  Force  had  similar  results.  The  Army  had  five  hits,  the  Navy  three  hits,  and 
the  Air  Force  four  hits.  What  is  interesting  is  the  distribution  for  utilization  under  level  of 
command.  The  MTF  command  teams  had  nine  hits,  the  TRICARE  regions  three  hits,  and 
the  executive  level  had  zero  hits.  This  points  to  a  requirement  and  importance  of  tracking 
utilization  specific  measures  at  the  point  of  care. 

Under  staffing  measures,  the  break  down  was  more  marked  under  services.  The 
Army  and  Air  Force  had  four  hits  and  the  Navy  only  one.  There  might  be  a  number  of 
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reasons  for  this,  but  one  cannot  assume  without  further  research.  Again,  looking  at 
staffing  from  the  level  of  command  variable,  one  notes  a  difference  in  hits.  The  MTF 
command  teams  requested  staffing  measures  eight  times,  the  TRICARE  regions  zero 
times,  and  the  executive  level  one  time.  This  may  lead  one  to  assume  that  staffing  ratios 
are  important  to  track  at  the  hospitals. 

Finally,  under  employee  satisfaction,  it  is  interesting  to  note  that  all  the  hits 
originated  from  questionnaires  at  the  MTF  level  of  command.  This  could  be  a  function  of 
heightened  awareness  of  employee  and  provider  satisfaction  at  the  hospital.  The  leaders 
in  these  organizations  are  normally  the  ones  that  hear  the  complaints  and  observe 
dissatisfaction  first  hand.  Provider  and  employee  satisfaction  measures  are  missing  in  the 
proposed  standard  set.  Dr.  Sears,  Executive  Director,  TMA  has  spoken  many  times  of  the 
importance  of  provider  satisfaction  within  the  MHS.  In  all  his  visits,  he  incorporates 
provider  panels  to  assess  provider  satisfaction  and  field  questions.  Provider  and  employee 
satisfaction  measures  are  critical  leading  indicators  of  customer  satisfaction  and  in  many 
cases  operational  effectiveness  (Berkow  &  Seth,  2000).  Based  on  the  results  from  the 
questionnaire,  TMA  should  consider  adding  some  provider  satisfaction  measures  to  the 
standard  set. 

Unfortunately,  other  than  portraying  the  results  and  ranking  the  top  five  requested 
performance  measures  and  the  17  performance  areas,  the  research  cannot  devise 
conclusions  from  these  data  due  to  some  constraints.  The  most  salient  concern  was  that 
the  MTFs  had  the  greatest  number  of  questionnaires;  therefore,  they  had  the  majority  of 
the  input.  A  majority  of  input  leads  to  the  greatest  number  of  hits.  Possibly,  that  is  why 
the  majority  of  recommended  measures  and  domains  are  hospital  specific. 
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Factor  Analysis 

Based  on  the  high  number  of  measures  that  demonstrated  agreement  among 
respondents  and  the  various  domain  areas  proposed  under  part  III,  the  researcher 
conducted  a  test  to  determine  areas  of  natural  agreement.  This  was  the  next  logical  step  to 
see  if  the  domains  or  performance  measurement  areas  selected  by  TMA  and  HA 
coincided  with  the  results  from  the  field.  As  stated  earlier  TMA  grouped  the  measures 
under  quality,  access,  cost,  health  outcomes,  and  special  interest. 

Factor  analysis  identifies  those  underlying  constructs  among  responses.  A  factor 
or  component  is  a  “construct,  a  hypothetical  entity,  a  latent  variable  that  is  assumed  to 
underlie  tests,  scales,  items... ’’(Kerlinger,  1986,  p.  569).  Factor  analysis  is  a  technique 
“designed  to  identify  factors,  or  dimensions,  that  underlie  the  relations  among  a  set  of 
observed  variables”  (Pedhazur  &  Schmelkin,  1991,  p.  66).  It  discerns  natural  areas  of 
agreement  among  the  respondents  depending  on  the  way  that  they  rated  the  items  on  the 
questionnaire.  In  an  effort  to  reduce  data  in  determining  item  correlations,  the  correlation 
matrix  can  be  factor  analyzed.  This  creates  factor  loadings  (estimate  of  the  relation 
between  the  item  and  the  factor/dimension)  which  are  then  appraised  to  identify 
meaningful  item  to  factor  correlations  (Pedhazur  &  Schmelkin). 

Two  iterations  of  Factor  Analysis  were  computed.  Factors,  dimensions,  or 
components  were  retained  for  analysis  if  they  had  an  eigenvalue  equal  to  or  greater  than 
1.00.  In  the  first  run,  11  components  were  identified  with  an  eigenvalue  of  1.00  or  more. 
The  total  variance  attributed  to  these  components  was  80.9%.  Seven  of  the  eleven 
components  were  identified  clearly.  The  other  four  components  had  measures  of  similar 
content  except  for  component  number  nine.  Loading  values  of  .5  or  greater  were  used  to 
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place  specific  items  within  the  selected  component.  The  prevailing  literature  states  that 
loading  values  of  .4  or  .5  are  considered  meaningful  (Pedhazur  &  Schmelkin,  1991).  In 
this  run,  two  items  were  not  considered  meaningful  under  any  of  the  1 1  components. 
They  were  nurse  full  time  equivalents  per  weighted  bed  day  and  JCAHO  grid  scores.  The 
highest  loaded  values  for  the  Nurse  FTE  item,  .496  and  .467  fell  within  the  professional 
competency  and  cost  components  respectively.  TMA  had  placed  it  is  under  the  cost 
domain. 

Table  14.  Factor  Analysis,  11  components 


Component 

Eigenvalue 

%  of  Variance 

Cumulative  % 

Customer  Satisfaction  (1) 

12.031 

25.598 

25.598 

Soft  measures 

Cost  (2) 

6.497 

13.824 

39.422 

Professional  Competency  (3) 

4.370 

9.298 

48.720 

Data  Quality  (4) 

2.694 

5.732 

54.452 

Customer  Satisfaction  (5) 

2.604 

5.540 

59.992 

Hard  measures 

Health  Status/preventive  (6) 

2.345 

4.989 

64.981 

Market  Share  (7) 

1.934 

4.115 

69.096 

Utilization/Cost/Enroll  (8) 

1.661 

3.533 

72.629 

Readiness  (9) 

1.514 

3.221 

75.850 

Utilization/Cost/  (10) 

1.233 

2.624 

78.474 

Utilization/Enroll  (11) 

1.159 

2.465 

80.939 

The  eleven  components  were  difficult  to  discern  due  to  a  number  of  overlapping 
underlying  constructs  and  concepts.  Based  on  the  eigenvalues,  the  researcher  cut  the 
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available  factors  down  to  three.  This  reduction  was  accomplished  with  guidance  from  Dr. 
Mangelsdorff,  Baylor  University,  to  try  and  reduce  the  components  further  and  to 
siriiplify  the  structure  and  analysis  (Dave  Mangelsdorff,  personal  communication,  March 
5, 2001).  To  properly  reduce  the  components,  the  eigenvalues  are  examined  to  determine 
the  most  noticeable  drop  in  value.  In  this  study,  a  scree  plot  was  prepared  to  give  a  visual 
representation  of  the  outcome.  The  scree  plot  below  exposes  the  “edge  of  the  cliff’  (S. 
Porter,  personal  communication,  March  3,  2001)  clearly.  Any  values  below  the  edge  of 
the  cliff  are  considered  superfluous. 

Figure  1.  Scree  Plot,  Factor  Analysis 


Components  with  eigenvalues  less  than  4.0  were  discarded.  The  second  iteration  was 
calculated  as  before,  except  that  the  researcher  set  a  limit  on  the  number  of  factors  or 
components  to  three. 
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Table  15.  Factor  Analysis,  3  components 


Component 

Eigenvalue 

%  of  Variance 

Cumulative  % 

Financial  (1) 

12.031 

25.598 

25.598 

Satisfaction  (2) 

6.497 

13.824 

39.422 

Quality  &  Operations  Efficiency  (3)  4.370 

9.298 

48.720 

Under  this  parameter,  the  cumulative  variance  attributed  to  the  three  components 
with  an  eigenvalue  over  four  is  48.7%.  Factor  loadings  above  .5  were  used  to  fit  the 
metrics  within  a  component.  Thirty-two  items  were  “relatively  ‘pure’  measures  of  their 
respective  factors”  (Kerlinger,  1986,  p.  575).  That  means  that  those  tests  were  highly 
loaded  in  one  factor  as  opposed  to  the  other  two  factors.  A  highly  loaded  test  exhibits  a 
good  fit  within  the  factor.  For  example,  laboratory  costs  had  a  factor  loading  of  .836 
under  factor  one  (financial)  versus  loadings  of .  177  and  .074  under  the  other  components. 
Therefore,  laboratory  costs  would  be  described  as  being  a  pure  measure  of  factor  one 
(financial). 

Table  16.  Number  “pure”  items  by  component 


Component  #  items  high  load 

total  measures 

%  pure 

Financial  (1) 

11 

14 

78.6 

Satisfaction  (2) 

09 

09 

100.0 

Quality  &  Operations  Efficiency  (3) 

12 

15 

80.0 

Measures  not  fitting  into 

any  Factor  (.5  <  loading) 

09 
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The  items  that  were  not  pure  had  at  least  two  loadings  that  were  relatively  similar.  As 
exhibited  in  the  table  above,  satisfaction  had  the  highest  percentage  of  pure  items  based 
on  the  measures  selected  with  a  loading  of  .5  or  greater.  The  results  above  point  to  high 
agreement  among  the  respondents  that  the  satisfaction  type  items/metrics  measured 
satisfaction  related  objectives.  Financial,  with  the  lowest  percentage,  had  more  items  as  a 
ratio  that  did  not  reveal  high  loading  factors,  when  compared  to  the  other  two 
components. 

This  may  point  to  slight  disagreement  among  the  respondents  as  to  what  those 
metrics  are  actually  measuring.  For  example,  based  on  the  responses,  nurse  FTE  per 
weighted  bed  could  fit  into  both  financial  and  quality  and  operations  efficiency 
domains/factors.  Other  examples  are  the  malpractice  claims  measures.  Are  the  underlying 
constructs  financial  or  quality/operations?  One  can  make  the  argument  for  either  one, 
depending  on  what  the  leadership  is  trying  to  measure.  The  following  table  breaks  down 
the  measures  with  similar  factor  loadings,  denoting  slight  disagreement  as  to  what 
component  or  domain  they  should  reside  in.  These  measures  could  become  candidates  for 
extensive  review  to  ensure  they  measure  what  was  intended,  based  on  management 
objectives.  In  addition,  these  measures  might  encourage  perverse  incentives  or 
unintended  changes  in  behavior. 

Table  17.  Items  with  similar  factor  loadings  (not  pure)  by  component 
Items  Financial  &  Health  (1)  Satisfaction  (2)  Quality  &  Ops  effic  (3) 

Capital  Costs  .570  -.102  .462 

Cost  per  Relative 

Weighted  Product  .423  -.136  .  .431 
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Items 

Financial  &  Health  (1) 

Satisfaction  (2) 

Quality  &  Ops  effic 

Cost  per  Visit- 
Weighted 

.462 

-.249 

.448 

Direct  Care  Market 
Share  (Inpatient) 

.236 

6.01E-02 

.254 

Direct  Care  Market 
Share  (Outpatient) 

.215 

.197 

.372 

Malpractice  Claims 
Filed 

.477 

.158 

.530 

Malpractice  Claims 
Paid 

.477 

.125 

.525 

MCSC  %  calls  answered 

In  two  minutes  .118 

.374 

.580 

Nurse  FTE  per 
Weighted  Bed  day 

.661 

6.72E-02 

.444 

Enrollees  seen  by 
PCM 

.557 

.386 

-6.57-02 

Wait  time  for 
Appointment 

.318 

.481 

l.OlE-02 

In  addition  to  the  list  portrayed  above,  nine  items  did  not  meet  the  .5  loading 
factor  threshold  score.  The  nine  metrics  are  1)  cost  per  relative  weighted  product,  2)  cost 
per  visit-weighted,  3)  direct  care  market  share  (inpatient),  4)  direct  care  market  share 
(outpatient),  5)  eligible  enrolled  at  MTF,  6)  emergency  room  visits,  7)  preventive  care 
measures,  8)  visits  per  provider  per  day,  and  9)  wait  time  for  appointment.  Five  of  the 
nine,  though,  had  loading  scores  above  .4.  They  are  1)  cost  per  relative  weighted  product, 
2)  cost  per  visit-weighted,  3)  eligible  enrolled  at  MTF,  4)  emergency  room  visits,  and  5) 
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wait  time  for  appointment.  Though  these  items  do  not  exhibit  meaningful  loading  factors, 
TMA  placed  them  into  the  cost,  access,  quality,  and  healthcare  outcome  domains. 

Table  18.  Items  with  loading  factors  less  than  .5  and  the  TMA  domains 


Items 

Domains 

Cost  per  Relative 

Weighted  Product 

Cost 

Cost  per  Visit- 

Weighted 

Cost 

Direct  Care  Market 

Share  (Inpatient) 

Access 

Direct  Care  Market 

Share  (Outpatient) 

Access 

Eligible  Enrolled  at  MTF 

Quality 

Emergency  Room  Visits 

Access 

Preventive  Care  Measures 

Health  Outcomes 

Visits  per  Provider  per  Day 

Cost 

Wait  time  for 

Appointment 

Access 

Though  the  above  items  are  not  significant  in  any  of  the  three  components,  they  are  still 
important  in  assessing  performance  and  goal  attainment.  For  example,  preventive  care 
measures  are  popular  metrics  of  population  health  and  wellness  improvements.  Based  on 
questionnaire  responses,  a  loading  factor  of  .499  or  lower  suggests  that  those  items  did 
not  fit  into  the  three  component  constraints.  Under  the  first  iteration  of  factor  analysis, 
where  the  eigenvalue  threshold  was  1.0  and  not  4.0,  all  nine  metrics  exhibited  factor 
loadings  greater  than  .5.  Five  of  the  nine  measures  described  above  fell  into  three  similar 
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components  in  the  first  test.  These  measures  are  cost  per  RWP,  cost  per  visit,  emergency 
room  visits,  eligible  enrolled  at  MTF,  and  visits  per  provider.  Those  three  components 
had  an  underlying  construct  relating  to  utilization.  This  construct  was  not  forced  into  the 
three  components  once  the  test  was  repeated  under  the  artificial  constraint  and  those 
measures  fell  out.  Not  fitting  into  an  existing  component  though,  does  not  connote 
deletion  from  the  proposed  standard  set.  It  represents  the  possibility  that,  a  decision  based 
on  the  scree  plot  to  reduce  the  components  increases  the  possibility  of  missing  some 
underlying  concepts  or  that  the  respondents  did  not  correlate  those  measures  to  other 
similar  ones.  It  basically  denotes  a  vacillation  among  respondents  if  the  components  are 
reduced  to  three  as  to  what  construct  those  measures  represent  best. 

Agreement  between  TMA’s  domain  selection  and  the  factor  analysis  results  is 
difficult  to  discern  based  on  the  different  number  of  factors/domains,  three  versus  five, 
and  the  different  construct  identifications.  Another  reason  for  this  difficulty  is  that  TMA 
has  satisfaction  measures  interspersed  between  quality  and  access.  Yet,  it  is  interesting  to 
see  the  relationship  between  the  factor-analyzed  respondents’  scores  and  TMA’s  domain 
selection.  The  following  side-by-side  table  illustrates  those  similarities  and  differences 
well. 

Table  19.  Comparison  between  factor  analysis  results  and  TMA  domains 


Performance  Measure 

FA 

TMA 

Satisfaction  with  Quality  of  Health  Care 

Satisfaction 

Quality 

%  Beneficiary  Satisfied  with  Customer  Service 

Satisfaction 

Quality 

Patient  Satisfaction  with  Access 

Satisfaction 

Access 

Satisfaction  with  Access  to  Healthcare 

Satisfaction 

Access 

Overall  Satisfaction  with  Clinic  Visit 

Satisfaction 

Quality 
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Performance  Measure 

FA 

TMA 

Patient  Satisfaction  with  Waiting  Times 

Satisfaction 

Access 

%  Eligible  Enrolled  to  MTF* 

N/A 

Quality 

Cost  per  Enrollee 

Financial 

Cost 

Ease  of  Making  Appointment  by  Phone 

Satisfaction 

Access 

Emergency  Room  Visits  per  1000  Enrollees* 

N/A 

Access 

Preventive  Care  Measures  (ex.  Pap  smears)* 

N/A 

Health 

Pharmacy  Cost  per  Member  per  Month 

Financial 

Cost 

Satisfaction  with  Access  to  Specialist 

Satisfaction 

Access 

Direct  Care  Market  Share  -  Outpatient  workload* 

N/A 

Access 

Wait  Time  for  Appointment  (TRICARE  Stds)* 

N/A 

Access 

%  of  Enrollees  Seen  by  their  PCM* 

Financial 

Quality 

Wait  Time  at  Appointment  (30  minute  standard) 

Satisfaction 

Access 

Visits  per  Provider  per  Day* 

N/A 

Cost 

Cost  per  Visit  Weighted* 

N/A 

Cost 

Dental  Readiness  (%  Class  1  and  2)* 

Quality/Ops 

Readiness 

%  Providers  Medical  Readiness  Tnd  &  Certified* 

Quality/Ops 

Readiness 

Cost  per  Bed  day 

Financial 

Cost 

Preventable  Admission  Rates— Prime  Enrollees* 

Financial 

Access 

Cost  per  RWP  (Relative  Weighted  Product)* 

N/A 

Cost 

Claims  Processed  within  30  Days* 

Quality/Ops 

Cost 

Visits  per  1000  Weighted 

Financial 

Cost 

Dental  Wellness  (%  Class  1)* 

Quality 

Readiness 

Direct  Care  Market  Share  -  Inpatient  workload* 

N/A 

Access 

MCSC  %  of  Calls  Answered  within  2  minutes* 

Quality/Ops 

Access 

Laboratory  Costs  per  Member/Month 

Financial 

Cost 

Radiology  Costs  per  Member/Month 

Financial 

Cost 

SADR  Completion  Rate* 

Quality/Ops 

Special 

SIDR  Completion  Rate* 

Quality/Ops 

Special 

Capital  Costs  per  Member  per  Month 

Financial 

Cost 
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Performance  Measure 

FA 

TMA 

Bed  days  per  1000  Enrollees 

Financial 

Cost 

MCSC  Phones  —  %  of  all  lines  busy* 

Quality/Ops 

Access 

Dispositions  per  1000  Enrollees 

Financial 

Cost 

Nurse  FTE  per  Weighted  Bed  Day 

Financial 

Cost 

Beneficiary  Grievances/ 1000  Enrollees 

Quality/Ops 

Quality 

RWPs  per  1000  enrollees 

Financial 

Cost 

Health  Status  (self  reported  on  SF  12 — mean)* 

Financial 

Health 

MEPRS  Completion  Rate* 

Quality/Ops 

Special 

%  of  AD  Members  with  Temp  Medical  Profiles* 

Quality/Ops 

Readiness 

Beneficiary  Appeals  per  1000  Claims 

Quality/Ops 

Quality 

JCAHO  Grid  Scores 

Quality/Ops 

Quality 

Malpractice  Claims  Paid  per  100  Physicians 

Quality/Ops 

Quality 

Malpractice  Claims  Filed  per  100  Physicians 

Quality/Ops 

Quality 

*Observed  differences  between  TMA  domain  and  the  factor 

As  reflected  above,  one  can  quickly  spot  similarities  and  differences.  The 
satisfaction  factor  relates  both  to  access  and  quality,  due  to  TMA’s  placement  of  both 
service  and  clinical  satisfaction  measures  into  access  and  quality  respectively.  Thus,  these 
are  not  considered  as  different,  since  the  underlying  idea  is  still  the  same.  Discounting 
those  observed  (*)  items  that  had  factor  loading  values  of  less  than  .5  (N/A  on  the  table 
above),  there  are  13  initial  items  with  unmatched  factors. 

There  are  a  number  of  reasons  for  this  phenomenon.  The  most  prominent  is  that 
under  calculations  based  on  only  three  components,  those  eigenvalues  above  4.0,  those 
measures  were  forced  into  a  factor.  Under  11  components,  the  first  factor  analysis  run, 
the  readiness,  special  (data  quality)  and  health  outcome  measures  as  defined  by  TMA, 
actually  fit  in  their  own  factor.  With  only  three  components  to  choose  from  they  were 
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placed  into  quality  and  operations  efficiency  for  the  readiness  measures  and  into  financial 
for  the  health  outcomes  measures. 

Yet,  if  one  decomposes  those  measures  under  readiness  and  special,  one  can  see 
their  relation  to  quality  and  operational  efficiency.  For  example,  a  high  state  of  dental 
wellness  or  readiness  could  equate  to  quality  care.  Further,  certified  and  trained  providers 
should  effectively  improve  the  quality  of  care.  The  respondents,  based  on  their  answers, 
related  (correlated)  these  measures  to  other  quality  type  metrics;  therefore,  the  underlying 
construct  for  these  measures  turns  out  to  be  quality. 

Those  special  (data  quality)  measures  were  placed  within  quality  and  operational 
efficiency  also.  This  is  logical.  Empirically,  the  same  concept  applies  to  data  quality  that 
applies  to  the  other  measures  of  quality.  The  respondents’  results  demonstrated  a 
correlation  between  those  data  quality  measures  and  the  other  quality  measures. 
Therefore,  the  data  quality  measures  had  high  factor  loadings,  above  .700,  under  the 
quality  and  operational  efficiency  component. 

Thus,  the  actual  unmatched  measures  dropped  from  13  to  six.  The  six  that  were 
not  readiness  or  special  related  were  1)  MCSC  %  of  calls  answered  within  2  minutes,  2) 
MCSC  phones  —  %  of  all  lines  busy  3)  claims  processed  within  30  days,  4)  preventable 
admission  rates— prime  enrollees,  5)  percent  of  enrollees  seen  by  their  PCM,  and  6)  health 
status.  Four  of  the  six  metrics  were  pure  measures  in  their  respective  factor.  They  are  1) 
MCSC  phones— percent  of  all  lines  busy  2)  preventable  admission  rates,  3)  claims 
processed,  and  4)  health  status.  These  measures  should  be  looked  at  to  ensure  they  are 
driving  the  desired  behavior  at  the  MTFs  and  the  regions.  In  these  cases,  there  is  little 
agreement  as  to  where  these  measures  should  fall  between  TMA  and  the  questionnaire 
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respondents.  For  example,  preventable  admission  rates  are  considered  an  access  issue  at 
TMA;  yet  in  the  field  it  becomes  a  measure  of  financial  importance.  This  may  effect  a 
change  within  the  operational  process  and  the  leadership  focus  at  the  MTF,  which  is  not 
anticipated  at  TMA. 

The  other  two  measures  were  not  pure.  Most  respondents  correlated  enrol  lees 
seen  by  PCM  to  other  financial  measures  and  MCSC  calls  answered  within  two  minutes 
to  other  quality/operations  efficiency  measures.  TMA,  though,  considers  MCSC  calls 
answered  an  access  metric  and  enrollees  seen  by  PCM  as  a  quality  measure.  MCSC  calls 
answered  within  two  minutes  exhibited  factor-loading  scores  of  .374  for  satisfaction  and 
.580  for  quality/operations  efficiency.  Satisfaction  is  related  to  access  measures; 
therefore,  the  difference  is  opinion  is  minimal.  On  the  other  hand,  enrollee  seen  by  PCM 
had  an  extremely  low  factor  loading  score  under  quality/operations  efficiency  (.0657)  as 
compared  to  .557  for  financial.  Thus  it  seems  that  the  respondents  correlated  this  metric 
to  other  financial  type  metrics  instead  of  quality  type,  metrics.  TMA  believes  that  if  a 
patient  is  seen  by  the  PCM,  clinical  quality  increases.  The  respondents  link  PCM 
treatment  of  a  patient  to  organizational  fiscal  health,  not  to  an  increase  in  the  quality  of 
healthcare  provision.  This  may  underlie  the  belief  that  providers  in  MTFs  are  qualified, 
professional,  and  will  care  for  those  non-enrolled  and  enrolled  patients  equally. 

Although  the  last  couple  paragraphs  concentrated  on  the  differences  between 
TMA  and  the  respondents,  the  interesting  point  are  the  strong  similarities.  Thirty-two 
performance  measures  were  empirically  located  within  factors  that  matched  TMA’s 
rational  selection  of  groupings.  That  equates  to  a  68%  match  using  only  three 
components  as  a  constraint  and  including  the  measures  with  loading  factors  under  the  .5 
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threshold  in  the  total.  Not  including  those  measures  that  did  not  meet  the  threshold  the 
result  becomes  84%.  Quality  measures  had  the  highest  level  of  agreement  at  80% 
(including  N/A)  and  89%  (not  including  N/A).  Even  with  the  untraditional  selection  of 
domains  to  match  the  tenets  of  healthcare,  the  match  between  the  field  and  TMA  is 
considerable.  That  may  be  the  case  in  point  within  the  MHS,  but  a  comparison  with  the 
private  sector  yielded  diverse  results. 

Gap  Analysis  between  the  Proposed  Standard  Set  and  the  Private  Sector 

To  complete  an  analysis  of  the  proposed  standard  metric  set  it  is  beneficial  to 
review  what  health  care  organizations  in  the  private  sector  are  doing.  The  gap  analysis 
was  conducted  to  identify  any  glaring  differences  and  similarities  between  the  private 
sector  and  the  MHS.  There  were  some  complications  during  this  process. 

First,  none  of  the  organizations  selected  domains  or  performance  measures  in  the 
same  manner.  For  example,  one  organization  was  studying  the  implementation  of  a 
balanced  score  card,  another  was  selecting  a  core  set  of  metrics  based  on  what  the 
leadership  thought  was  important,  and  another  tailored  their  metrics  to  customer  needs. 
The  thread  that  tied  all  those  organizations  though,  was  a  system  that  connected  the 
performance  measures  to  management  objectives.  The  objectives  were  either  short-term 
or  long-term,  depending  on  the  organizational  plan.  Two  organizations,  in  addition  to 
long-term  plans,  created  metrics  to  measure  performance  for  short-term  (one-year)  goals. 
These  short-term  plans  were  organized  to  support  major  strategic  goals  within  functional 
areas.  In  addition,  the  short-term  plan  was  used  to  create  objectives  for  senior  managers 
to  incorporate  into  their  yearly  performance  appraisals.  The  inclusion  of  objectives  in 
performance  appraisals  is  similar  to  the  Army’s  support  form. 
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A  second  complication  was  that  many  organizations  had  reports  that  were  not 
integrated.  A  case  similar  to  the  MHS  where  integration  based  on  management  objectives 
is  lacking.  These  establishments  had  hundreds  of  performance  measures  that  were  formed 
at  all  levels  within  their  organizations. 

Third,  because  the  measures  were  created  without  a  coordinated  effort,  domains 
and  supporting  goals  were  difficult  to  determine.  Fourth,  in  some  cases,  the  researcher 
may  not  have  received  all  the  pertinent  information  from  the  organization.  Regardless  of 
these  challenges,  the  information  provided  by  the  organizations  was  extremely  useful  to 
eonduct  a  cursory  gap  analysis. 

The  first  step  was  to  distinguish  the  domains  or  performance  measurement  areas 
used  by  the  organizations,  determine  those  counts  by  organization,  and  count  the  number 
of  measures  within  each  area  (appendix  AC  &  AD).  Appendix  AC  shows  the  total 
number  of  performance  measures  reviewed  at  the  senior  level  of  each  organization.  The 
average  number  of  performance  measures  for  the  private  sector  was  39.  These  figures 
range  from  a  low  of  19  to  a  high  of  62.  The  recommended  number  of  performance 
measures  for  a  health  plan  was  33  and  a  health  system  40  (Berkow  &  Seth,  2000).  The 
MHS  with  47  measures  in  the  proposed  standard  set  falls  within  one  standard  deviation, 
a  =  14.55,  from  the  private  sector  mean  (39).  In  total  quantity  terms,  the  difference  is  not 
significant. 

Appendix  AD  breaks  out  the  domains  by  organization  with  the  number  of 
measures  per  domain.  As  one  can  see,  the  private  sector  domain  names  are  not  similar  to 
the  MHS’s.  This  is  predominately  due  to  their  practice  of  tying  a  domain  to  major 
strategic  goals.  Though,  there  were  no  instances  that  the  organizations  had  implemented 
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the  balanced  score  card  (BSC)  system,  their  selection  of  domains  represents  an  approach 
similar  to  BSC.  Those  organizations  selected  areas  from  different  business  processes  to 
provide  a  detailed  picture  of  performance.  According  to  the  Berkow  and  Seth  (2000), 
health  system  executives  are  adopting  dashboard  type  metric  tools  that  incorporate 
various  areas  of  interest.  To  depict  a  balanced  type  approach,  the  MHS  related  their 
domains  to  the  three  healthcare  tenets  portrayed  in  the  iron  triangle  (quality,  access,  and 
cost).  Though  these  labels  are  different,  the  MHS  also  reviews  results  from  different 
operational  perspectives  that  are  similar  in  nature.  A  major  similarity  is  the  popularity  of 
specific  performance  measurement  areas  among  the  organizations. 

Table  20.  Domains  with  the  highest  number  of  measures  by  type  of  organization 


Organization  Type 

Domain 

#  Measures 

Health  System 

Financial 

28 

Health  System 

Financial 

22 

Health  Plan 

Financial 

08 

Health  Plan 

Quality 

15 

Health  Plan 

Performance 

11 

Health  Plan  (HCAB) 

Financial 

10 

Hospital  (HCAB) 

Operational 

11 

HMOs 

Financial 

20 

MHS 

Cost 

15 

Though,  all  the  organizations  include  various  performance  areas,  undoubtedly, 
financial  measures  are  still  the  most  popular.  According  to  Kaplan  and  Norton  (1996), 
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historically  the  measurement  system  for  business  is  financial.  The  results  above  show  that 
both  the  MHS  and  the  private  sector  concentrate  on  fiscal  measures.  The  similarity  is  the 
requirement  to  remain  fiscally  viable.  The  private  sector’s  interest  is  revenue  growth  and 
expense  reduction,  while  the  MHS  concentrates  on  controlling  costs  to  remain  within 
budgetary  parameters.  When  the  numbers  of  measures  per  domain  are  aggregated  to 
include  all  the  private  sector  organizations,  one  obtains  some  interesting  proportions 
(Appendix  AE). 

The  highest  proportion  for  both  the  private  sector  and  the  MHS  was  financial.  The 
financial  proportion  of  the  total  number  for  the  private  sector  was  39%  and  32%  for  the 
MHS.  The  second  highest  for  the  private  sector  was  quality  which  included  both  clinical 
and  service.  Clinical  quality  was  much  greater  than  service,  17%  proportion  to  total 
versus  2%.  Further  more,  the  private  sector’s  customer  satisfaction  measures  equated  to 
only  6%  of  the  total. 

The  MHS’s  second  highest  was  access  (28%)  which  relates  most  closely  to 
service  quality  and  customer  satisfaction.  Based  on  proportions,  the  private  sector 
concentrates  more  on  those  clinical  quality  measures  and  less  on  the  soft  customer 
satisfaction  measures.  Examples  of  clinical  quality  measures  include  unplanned  return  to 
specialty  care  and  operating  room  rates,  patient  fall  with  injury  rate,  number  of  serious 
adverse  drug  events,  and  acute  care  readmission  rate.  The  MHS  on  the  other  hand,  was 
more  interested  on  those  soft  customer  service  measures,  which  are  normally  self- 
reported.  Interestingly,  the  MHS  respondents  (part  II)  ranked  six  of  those  soft  customer 
support  measures  (access)  in  the  top  10  most  applicable  for  the  standard  set. 
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Finally,  the  number  that  begs  to  be  reviewed  is  the  proportion  of  employee 
satisfaction  measures  to  the  total  within  the  private  sector.  Including  one  workplace 
measures,  the  number  is  7%  as  compared  to  the  MHS’s  0%.  According  to  the  MHS 
questionnaire  respondents,  the  top  measure  that  should  be  included  in  the  proposed 
standard  set  is  an  employee  satisfaction  measure.  In  addition,  the  executive  director  of 
TMA  takes  special  care  to  include  employee  satisfaction  issues  on  his  trips.  The 
importance  of  this  measure  is  its  ability  to  predict  future  financial  health  and  customer 
satisfaction.  The  metric  working  group  should  review  the  possibility  of  adding  selected 
employee/provider  satisfaction  measures  within  the  standard  set.  Agreement  is  strong 
among  the  questionnaire  respondents,  the  Executive  Director  of  TMA,  and  the  private 
sector.  Differences  and  similarities  are  obvious  to  the  naked  eye;  the  point  now  is  to 
quantify  those  differences. 

The  next  step  in  the  analysis  is  to  establish  a  variance  between  the  domains  of  the 
private  sector  and  the  MHS  (Appendix  AF).  This  process  was  tricky,  because  to  properly 
quantify  differences,  the  researcher  had  to  manipulate  the  performance  measures  to  insure 
orange-to-orange  comparisons.  As  one  can  detect  in  the  appendix,  the  private  sector 
proportions  were  slightly  modified  to  ensure  that  the  underlying  constructs  between 
domains  matched. 

The  largest  variances  were  noted  under  access  and  cost.  The  MHS  has  16%  more 
access  type  measures  (satisfaction  and  customer  oriented  operational)  than  the  healthcare 
organizations  in  the  private  sector.  On  the  other  side  of  the  coin,  the  MHS  has  15%  less 
cost  (financial  and  fiscal  operational)  measures  than  the  private  sector.  The  difficulty  of 
analyzing  these  data  was  determining  what  measures  fit  in  which  domains.  Because  the 
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MHS  domains  do  not  fit  uniformly  with  the  private  sector,  this  reshuffling  probably 
introduced  additional  error  into  the  analysis  process.  The  results  under  the  general 
variance  column  in  appendix  AF  are  the  best  possible  representations  of  the  raw  data. 

The  results  lead  one  to  conclude  that  certain  differences  exist,  the  magnitude  of  those 
differences,  as  represented  in  the  variance  column,  must  be  interpreted  with  caution.  Now 
that  the  general  differences  are  identified,  what  are  some  of  the  similarities  in  measures? 

Appendix  AG  points  out  all  the  similar  performance  measures  between  the  MHS 
and  each  organization.  The  results  were  lower  than  anticipated  and  with  further  analysis 
some  interesting  themes  were  obtained. 

Table  21.  Number  of  common  performance  measures  with  the  MHS 


Organization  Name 

#  Similar  Measures 

%  of  Total  that  are  Similar 

Johns  Hopkins 

8 

14 

INOVA 

9 

15 

United  Health 

3 

16 

Humana 

4 

15 

Anthem 

6 

19 

Health  Plan  (HCAB) 

8 

20 

Hospital  (HCAB) 

3 

09 

American  Association 
of  Health  Plans  (AAHP) 

3 

09 

The  results  of  the  Health  Care  Advisory  Board  study  for  health  plans  yielded  the 
highest  percentage  of  measures  similar  to  the  MHS.  This  makes  sense,  because  TMA’s 
proposed  metric  set  has  a  health  plan  focus.  The  metric  set  is  made  up  predominately 
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with  those  measures  found  in  private  sector  health  plans.  The  low  percentage  with  the 
measures  contained  in  the  AAHP  document  is  based  upon  a  high  concentration  of 
financial  measures.  Eighty-two  percent  of  AAHP’s  measures  were  financial. 

When  compared  to  the  HCAB’s  hospital  measures,  the  similarity  was  low  with 
only  nine  measures.  The  proposed  standard  set  is  a  template  and  could  not  include  all 
hospital  level  metrics.  The  MTF  respondents’  requests  (part  III)  for  hospital  specific 
additional  metrics  and  a  low  association  between  the  standard  set  and  HCAB’s  hospital 
template  indicate  a  possible  weakness  in  this  standard  set. 

Those  similar  health  plan  measures  were  distributed  among  TMA’s  cost,  quality, 
and  access.  Four  of  the  nine  fell  under  cost,  three  under  quality,  and  two  under  access. 
The  HCAB  allocated  these  measures  under  utilization,  clinical  quality,  and  service 
quality.  Six  measures  were  under  utilization,  one  under  clinical  quality,  and  one  under 
service  quality.  Though  the  measures  are  similar  in  type,  their  application  based  on  the 
underlying  construct  is  different.  The  table  below  reflects  the  specific  domains,  to  include 
the  results  of  factor  analysis. 

Table  22.  Common  measures  between  HCAB  (health  plans)  and  the  MHS 


Measure 

MHS  Domain 

FA  component 

Health  Plan  Domain 

Beds  occupied  per  1000 

Cost 

Cost 

Cost 

#  ER  visits 

Access 

N/A 

Utilization 

#  Enrolled 

Quality 

N/A 

Utilization 

Pharmacy  expenses  PMPM 

Cost 

Cost 

Utilization 

Outpatient  visits  per  1000 

Cost 

Cost 

Utilization 

Readmission  rates 

Access 

Cost 

Clin  Quality 

Table  Continues 
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Measure 

MHS  Domain 

FA  component 

Health  Plan  Domai 

Physician  encounters  PMPM 

Cost 

N/A 

Utilization 

#  Beneficiary  grievances 

Quality 

Quality 

Serv  Quality 

The  difference  between  access  and  quality  (MHS)  and  utilization  (health  plans)  is 
remarkable  to  point  out.  In  the  MHS  access  measures  measure  availability  of  resources  . 
for  our  beneficiaries.  Access  is  normally  a  concept  the  MHS  wants  to  maximize, 
increasing  its  availability.  Utilization  on  the  other  hand  in  a  health  plan  is  something  that 
the  managers  want  to  decrease,  especially  if  capitated.  Note  ER  visits  and  readmission 
rates.  It  could  be  eloquently  argued  that  the  TMA  working  group  has  misplaced  those  two 
measures,  possibly  creating  unintended  behaviors.  The  factor  analysis  results 
(respondents)  for  ER  visits  put  it  under  the  financial  component  at  .436  with  negligible 
results  in  the  other  two  components.  The  health  plan  recommended  measures  (HCAB) 
considers  ER  visits  under  the  utilization  component/domain.  The  lower  the  metric  results, 
the  more  effective  the  medical  management  technique,  reducing  expenses  and  the 
medical  loss  ratio.  Under  TMA’s  access  domain,  the  ER  visit  metric  seems  to  encourage 
ER  use,  without  truly  decomposing  and  defining  the  metric  further. 

The  same  can  be  argued  for  readmission  rates.  Why  would  the  MHS  place 
readmission  rates,  a  measure  it  should  strive  to  reduce,  under  access?  The  private  sector’s 
placement  under  clinical  quality  is  more  appropriate  because  it  measure  effective  health 
care  delivery  during  the  first  admission.  A  readmission  for  a  similar  procedure  points  to 
reduced  quality  of  care.  Thus,  this  measure  is  a  leading  indicator  of  clinical  quality,  not 
access.  It  is  a  lagging  statistical  measure  of  access,  with  possible  unintended  results.  The 
other  six  measures  match  in  general  underlying  construct  terms.  The  importance  in  this 
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example  is  that  even  if  the  measures  are  similar,  the  purposes  based  on  the  underlying 
constructs  may  differ.  The  incentives  produced  by  the  metrics  may  not  be  the  same. 
Though  the  measures  are  the  same,  employee  behavior  will  differ  from  organization  to 
organization  depending  on  the  incentive. 

Those  purposes  and  incentives  are  normally  in  line  with  management  objectives 
and  goals  reflected  in  the  strategic  plan.  Therefore  it  is  important  to  understand  the 
underlying  constructs  within  each  measure  to  ensure  the  construct  relates  to  the 
objectives.  This  complicates  benchmarking  operations  between  the  private  sector  and  the 
MHS.  On  the  surface  the  metrics  may  look  similar,  but  the  purposes  for  each  may  be 
different. 

Therefore,  before  the  MHS  undertakes  additional  major  benchmarking  projects, 
specifically  when  comparing  to  the  private  sector,  it  is  beneficial  to  study  the 
relationships  of  those  underlying  constructs.  Based  on  the  gap  analysis,  the  MHS  and  the 
private  sector  organizations  do  exhibit  a  noteworthy  difference.  The  differences  exist 
among  domains,  performance  measurement  types,  their  design  and  implementation. 

Because  the  MHS  has  a  number  of  different  strategic  plans,  goals,  and 
management  objectives,  metric  design  and  implementation  does  not  map  back  to  specific 
management  objectives  agreed  upon  by  senior  management.  The  Army  with  its  balanced 
score  card  initiative  is  creating  a  system  that  most  closely  mirrors  the  private  sector,  but 
unfortunately,  it  applies  specifically  to  the  Army  and  not  the  MHS  as  a  whole.  With  this 
in  mind,  if  the  MHS  wants  to  institute  a  strategic  management  system  with  applicable 
metrics,  the  balanced  scorecard  is  presently  the  tool  “du’  jour”  in  the  private  sector. 
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Summary 

This  project/study  has  both  academic  and  corporate  application.  The  OSD’s 
senior  leaders  are  asking  the  MHS  to  justify  its  $17  billion  budget.  As  the  DHP  grows, 
weapons  systems  are  cut.  This  unfortunately  puts  the  MHS  under  the  microscope.  To 
ensure  improved  business  practices  reducing  the  overall  costs,  the  three  services  and 
TMA  are  adopting  changes  in  performance  management  processes. 

The  results  of  this  study  were  surprising.  Though  the  MHS  is  compartmentalized, 
specifically  by  service,  the  results  of  the  internal  MHS  questionnaire  exhibit  no  major 
significant  difference.  Of  the  13  general  metric  utilization  measures  only  one  exhibited  a 
significant  difference  of  opinion  among  the  respondents.  This  points  to  general  agreement 
among  various  military  officers  from  different  services  as  to  their  understanding  of 
general  strategic  performance  measurement  topics.  Agreement  at  this  level  denotes  an 
equal  playing  field  among  the  respondents.  Even  though  different  service  strategic 
positions  and  direction  complicate  the  landscape,  the  respondents  are  aware  of  these 
factors  and  agree  that  initiatives  to  simplify  the  system  be  implemented.  The  time  is  ripe 
for  change  implemented  with  leadership  buy-in. 

The  proposed  standard  metric  set  is  an  excellent  starting  point  to  standardize  and 
simplify  performance  measurement.  The  respondents  exhibited  a  high  degree  of 
agreement  with  the  proposed  measures  within  the  set.  This  supports  the  argument  that  a 
working  group,  properly  represented  from  each  service,  can  produce  a  tool  for  use 
throughout  the  MHS.  With  minor  modifications,  the  proposed  standard  set  fills  the  need 
for  a  metrics  template  and  executive  management  tool.  Adding  leading  indicators  like 
employee  or  provider  satisfaction  measures  to  this  set  would  improve  predictive  power. 
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Within  the  proposed  set  six  measures  represented  significant  disagreement  as  to 
their  value.  In  addition,  the  respondents  rated  approximately  six  more  measures  lowest, 
outside  one  standard  deviation,  on  the  scale.  Once  analyzed  these  measures  may  be 
excluded  in  future  iterations  of  the  standard  set.  Furthermore,  a  number  of  hospital 
specific  and  employee  satisfaction  type  measures  should  be  considered  for  inclusion  in 
the  standard  set. 

Based  on  factor  analysis,  a  review  of  the  domain  labels  added  value.  Possibly 
changing  them  from  the  tenets  of  healthcare  to  performance  measurement  areas  that 
relate  better  to  corporate  goals,  priorities,  and  management  objectives.  What  does  access 
mean?  One  of  its  components,  customer  satisfaction  is  simpler  to  qualify  and  define  as  a 
construct.  Among  other  factors,  quality  equates  to  both  clinical  and  service  quality, 
extricating  those  into  separate  domains  will  reduce  confusion  among  employees. 

Finally,  some  performance  measures  in  the  set  may  impel  perverse  incentives.  A 
testing  period  prior  to  full  implementation  will  identify  those  measures  that  drive 
unintended  behaviors.  Misunderstanding  senior  leader  objectives  increases  the  likelihood 
of  a  confounding  application  of  the  measures.  That  also  further  supports  simplification  of 
domains  and  measurement  tools. 

The  gap  analysis  conducted  with  the  private  sector  exposed  the  pertinent 
differences  between  the  MHS  and  those  organizations.  The  most  salient  was  the  domain 
differences,  which  further  supports  a  review  of  TMA’s  domains  and  the  assignment  of 
certain  measures  under  each.  Although  the  percentages  of  similar  measures  between  the 
MHS  and  the  private  sector  were  low,  the  underlying  constructs  were  generally 
analogous.  Both  types  of  organizations  focus  on  fiscal,  customer  satisfaction,  and  quality 
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type  measures.  Benchmarking  MHS  operations  and  results  should  be  conducted  sparingly 
to  ensure  the  differences  are  sifted  out.  Unfortunately,  though  the  concepts  are  similar, 
different  measures  in  these  organizations  measure  those  constructs.  The  MHS  cannot 
assume  that  because  a  metric  is  similar  it  measures  comparable  performance  results  in 
support  of  selected  management  objectives. 

From  the  academic  perspective,  this  study  did  not  just  educate  the  researcher,  but 
also  the  various  command  teams  and  executive  elements  participating  in  the  study.  The 
goal  is  to  energize  the  leaders  to  ask  questions,  basically  become  a  catalyst  for  further 
analysis  on  the  application  of  metrics  under  the  tenets  of  health  care  in  the  MHS.  In 
addition,  it  gave  them  an  avenue  to  provide  input  to  TMA  as  part  of  the  process. 

In  conclusion,  the  proposed  standard  set  is  a  suitable  instrument  that  can  be  used 
effectively  as  a  template  throughout  the  MHS.  It  is  suitable  due  to  its  acceptance  within 
the  MHS  based  on  the  responses  to  the  questionnaire.  Several  respondents  echoed  that  the 
set  was  “very  inclusive”  and  represented  “a  good  start”  (Respondent  questionnaires, 
2001).  This  set  is  the  first  step  towards  meeting  the  guidelines  set  forth  by  the  ASD  (HA) 
and  the  DMOC  to  streamline  and  improve  the  MHS’s  performance  measurement  and 
reporting  system.  Based  on  general  agreement,  with  some  modifications,  it  is  a  tool 
useful  to  senior  leadership  to  track  key  management  areas  and  goal  attainment.  At  the 
operational  (MTF  and  Region)  level  this  set  will  provide  a  comprehensive  framework  as 
a  starting  point  to  enhance  decision-making  in  accordance  with  established  goals. 

Recommendation  for  Future  Study. 

This  study  opens  up  a  number  of  possibilities  for  further  research.  Integration  is 
the  main  goal  of  the  optimization  plan;  therefore,  most  of  the  efforts  of  senior  leaders  in 
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the  MHS  should  be  geared  towards  integrating  our  three  services’  health  care  systems 
and  strategic  positions.  Specifically  in  an  area  that  requires  unity  of  effort: 
implementation  of  the  MHS  strategic  plan.  Future  work  could  be  accomplished  to  assist 
in  MHS  integration,  by  bringing  the  services  to  the  table  to  at  least  agree  on  a 
corporate/enterprise  direction. 

This  study  was  broad  in  breadth  in  an  effort  to  identify  any  differences  in 
definition  or  thought  process  among  the  services,  levels  of  command,  and  professions 
within  the  MHS.  In  addition,  a  cursory  gap  analysis  with  the  private  sector  organizations 
assisted  in  identifying  major  differences/gaps.  Follow  on  studies  could  study  performance 
measurement  from  a  more  micro  perspective.  For  example,  what  additional  metrics  are 
required  at  the  hospital  level  and  regional  level,  should  the  soft  (self  reported)  measures 
be  deleted,  what  perverse  behavior  are  metrics  with  low  factor  loading  driving,  and  how 
to  correct  this.  Also,  are  the  domains  appropriate  since  they  are  not  specifically  tied  to 
management  objectives,  is  metric  organization  within  each  domain  accurate,  based  on  the 
factor  analysis  what  can  be  done  to  align  the  TMA  and  the  field  for  those  differences,  and 
should  the  MHS  benchmark  with  the  private  sector. 

This  study  may  be  the  first  to  incorporate  MTF  command  teams  and  regional 
views  in  executive  making  decisions  at  the  strategic  level.  Metrics  is  a  “hot”  issue  in 
business  today  enabling  organizations  to  change  employee  behavior  to  accomplish  long¬ 
term  goals  and  meet  management  priorities.  Any  future  study  that  advances  the  corporate 
body  of  knowledge  in  regard  to  metrics,  integrated  health  care,  and  strategic  plans  will 
benefit  the  enterprise. 


Metrics 


82 


Reference  List 

Balm,  G.  (1996).  Benchmarking  and  gap  analysis:  What  is  the  next  milestone? 
Benchmarking  for  Quality  Management  &  Technology,  3(4),  28-33. 

Berkow,  S.,  &  Seth,  E.  (2000).  CEO  Dashboards:  Performance  measures  for  the 
new  health  care  economy.  Washington  DC:  Advisory  Board  Company. 

Birchard,  B.  (1995).  How  innovative  companies  really  use  the  new  metrics. 
CFO,  October,  43-51. 

Braun,  B.,  Koss,  R.,  &  Loeb,  J.  (1999).  Integrating  performance  measure  data 
into  the  Joint  Commission  Accreditation  process.  Evaluation  and  Health  Professions, 
22(3).  283-298. 

Brown,  M.  (1996).  Keening  Score:  Using  the  right  metrics  to  drive  world-class 
performance.  New  York:  AMACOM  Books,  American  Management  Association. 

Clancy,  CM.  (1997).  Ensuring  health  care  quality:  an  AHCPR  perspective. 
Clinical  Therapeutics,  19(6),  1564-1571. 

Cooper,  D.,  &  Schindler,  P.  (1998).  Business  Research  Methods  (6^**  ed.).  New 
York:  Irwin/McGraw-Hill. 

Cowan,  M.  (2001).  Washington  Perspectives.  Symposium  conducted  at  the 
American  College  of  Healthcare  Executives  Congress,  Chicago,  Illinois. 

Dowdy,  S.,  &  Wearden,  S.  (1991).  Statistics  for  Research  (2"'’  ed.).  New  York: 
John  Wiley  &  Sons. 

Dyck,  D.  (1996).  Gap  analysis  of  health  services:  Client  satisfaction  surveys. 
AAOHN  Journal.  44(11),  541-549. 


Gap  Analysis.  (2000).  [on-line].  Available:  http://www- 
mmd.eng.cam.ac.uk/people/ahr/dstools/choosing/gapana.htm 


Metrics 


83 


Harbour,  J.  (1997).  The  Basics  of  Performance  Measurement.  Portland,  OR: 
Productivity  Press. 

Hatcher,  L.  (1994).  A  Step-bv-Step  Approach  to  Using  SAS  System  for  Factor 
Analysis  and  Structural  Equation  Modeling.  Cary,  NC:  SAS  Institute  Inc. 

Headley,  D.,  &  Choi,  B.  (1992).  Achieving  service  quality  through  gap  analysis 
and  basic  statistical  approach.  Journal  of  Services  Marketing,  6(1). 

Health  Care  Advisory  Board.  (2000)  CEO  Dashboards.  [15  paragraphs].  Advisory 
Board.  Washington  DC.  [On-line  serial]  3392.  Available  HTTP. 

WWW. advisory.com/me... /basecontent.asp?contentid=3392&collectionid=85&program= 
Jennings,  B.,  &  Staggers,  N.  (1997).  The  hazards  of  outcomes  management. 
Journal  of  Outcomes  Management.  4(1).  18-23. 

Jennings,  B.,  &  Staggers,  N.  (1999).  A  provocative  look  at  performance 
measurement.  Nursing  Administration  Quarterly.  24(1),  17-30. 

Joint  Commission  on  Accreditation  of  Healthcare  Organizations  (JCAHO), 
(1993).  The  Measurement  Mandate.  Oakbrook  Terrace,  IL:  The  Joint  Commission  on 
Accreditation  of  Healthcare  Organizations. 

Kaplan  R.,  &  Norton  D.  (1993).  Putting  the  balanced  scorecard  to  work.  Harvard 
Business  Review.  71(5),  134-147. 

Kaplan  R.,  &  Norton  D.  (1996).  Using  the  balanced  scorecard  as  a  strategic 
management  system.  Harvard  Business  Review.  74(1),  75-85. 


Metrics 


84 


Kaplan  R.,  &  Norton  D.  (1996).  The  Balanced  Scorecard.  Boston,  MA:  Harvard 
Business  School  Press. 

Kaplan  R.,  &  Norton  D.  (1996).  Having  trouble  with  your  strategy?  Then  map  it. 
Harvard  Business  Review.  78(5),  167-176. 

Kerlinger,  F.  (1986).  Foundations  of  Behavioral  Research  (3"^^  ed.).  Fort  Worth, 
TX:  Harcourt  Brace  Jovanovich  College  Publishers. 

Kirk,  R.  (1999).  Statistics:  An  Introduction.  Fort  Worth,  TX:  Harcourt  Brace 
College  Publishers. 

Mangione,  T.  (1995).  Mail  Surveys.  Improving  the  Quality.  London,  England: 
Sage  Publications. 

McGlynn,  E.  (1998).  Choosing  and  evaluating  clinical  performance  measures. 
Joint  Commission  Journal  on  Quality  Improvement,  24(9),  470-479. 

Natarajan,  R.  (1999).  Continuous  improvement  of  service  operations:  Application 
of  service  template.  Total  Quality  Management,  10(6),  877-886. 

Normand,  S.,  McNeil,  B.,  Peterson,  L.,  and  Palmer  R.,  Eliciting  expert  opinion 
using  the  Delphi  technique:  Identifying  performance  indicators  for  cardiovascular 
disease.  International  Journal  of  Quality  in  Healthcare,  10(3),  247-260. 

Nunnaly,  J.  (1978).  Psychometric  Theory  (2"“  ed.).  New  York,  NY:  McGraw-Hill. 
Pedhazur,  E.  &  Schmelkin,  L.  (1991).  Measurement.  Design,  and  Analysis:  An 
Integrated  Approach.  Hillsdale,  NJ:  Lawrence  Erlbaum  Associates,  Inc. 

The  Price  Waterhouse  Change  Integration  Team.  (1995).  Better  Change.  Chicago, 


IL:  Irwin  Professional  Publishing. 


Metrics 


85 


Rand  Report  on  the  Merits  and  Feasibility  of  a  Joint  Command.  (2001). 
Unpublished. 

Schultz,  J.  &  Bowers,  J.  (1997).  Bayer’s  new  work  metrics  align  vision  and 
values.  Journal  of  Compensation  and  Benefits.  12(6),  36-42. 

Spector,  P.  (1992).  Summated  Rating  Scale  Construction.  Newbury  Park,  CA: 
Sage  Publications. 

U.S.  General  Accounting  Office.  (1999).  Defense  Health  Care:  Tri-Service 
strategy  needed  to  justify  medical  resources  for  readiness  and  peacetime  care. 


GAO/HEHS-00-10. 


Metrics 


86 


Appendix  A 
List  of  Metric  Sets 


Performance  Measurement 
System 

Proponent 

User  /  Management  Focus 

DHP  Performance  Contract 

OASD(HA) 

USD,  ASD(HA) 

TRICARE  Operational 
Performance  Statement  (TOPS) 

TMA  HPA&E 

TMA,  OASD(HA),  Services, 
TRICARE  Regions,  MTFs 

MHS  Measures  of  Performance 

OASD(HA)/ 
TMA  HPA&E 

DMOC,  USD(P&R),  ASD(HA), 

TMA  Executive  Director,  SGs 

MHS  Executive  Summary 

OASD(HA) 

USD(P&R),  DoD  PA&E 

Deliver  The  Promise 

TMA  HPA&E 

USD(P&R),  TMA,  ASD(HA),  SGs 

Population  Health  Operational 
Tracking  and  Optimization 
(PHOTO)  System 

TMAIM 

PCM  Teams,  MTF  Leadership, 
Intermediate  Commands,  TRICARE 
Regions,  OASD(HA),  TMA 

Several  other  performance  reports,  management  tools,  and  data  sets  are  currently  in  use 
within  the  MHS.  While  many  of  these  products  are  not  by  definition  metric  systems,  they 
have  listed  below  as  part  of  this  inventory. 


Product 

Region  Report  Card:  Analysis  of 
Leakage  to  the  MCS  Contracts 

TMA  RM 

TMA  Financial  Performance 
“Wall  Street  Report” 

TMA  RM 

Regions  At-A-Glance 

TMA  Ops 

Region  Status  Report 

TMA  Ops 

TRICARE  Prime  Enrollment 
Report 

TMA  Ops 

Template  Analysis  Tool 

CHCS  Access  Ad  Hoc  Tool 

Appointment  WIPT  Metrics 

Annual  HEDIS  Report 

Annual  Quality  Management 
Report 

TMA  Ops 

TRICARE  Consumer  Reports 

TMA  HPA&E 

Private  Sector  Care  Report 

USAF 

Provider  Perspectives 

Region  3 

User  /  Management  Focus 


TMA  RM,  Regions 


Senior  Executive  Leadership 


TMA,  OASD(HA),  Services, 
TRICARE  Regions,  MTFs 


TMA,  OASD(HA),  Services, 
TRICARE  Regions,  MTFs 


TMA,  OASD(HA),  Services, 
TRICARE  Regions,  MTFs 


MTF  Commander 


PCMs,  MTF  Leadership 


COMPARISON  OF  MILITARY  HEALTH  SYSTEM  (MHS)  PERFORMANCE  MEASUREMENT  SYSTEMS 
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METRIC  COMPARISON  SHEET 
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Wait  Time  for  Appointment  M5.  MTF  Prime  Enrollees  MTF  Prime  Enrollees 

Meeting  Appointment  Meeting  Appointment 

Waiting  Standards  (Wait  Waiting  Standards  for 

Time  for  Appt)  (DHP  Urgent  and  Routine 

Performance  Contract)  Appts  (Wait  Time  for 


Pharmacy  Cost  Direct  Care  Pharmacy 

(PMPM/PMPT) _ I _ I _ I _ _  Costs _ 


Dl.  Patient  Satisfaction  with 
Quality  of  Oral  Health 
Care  at  the  DTP 
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Standard  Metric  Set 


Appendix  C 


Presented  byTenent 


Tenent 

I.  Access 


II.  Quaiitv 


III.  Cost 


IV.  Readiness 


Performance  Measure 


1  Wait  Time  at  Appointment 

2  Wait  Time  for  Appointment 

3  Patient  Satisfaction  with  Access 

4  Patient  Satisfaction  with  Waiting  Times 

5  Satisfaction  with  Access  to  Healthcare 

6  Satisfaction  with  Access  to  Specialist 

7  Ease  of  Making  Appointment  by  Phone 

8  MCSC  Percent  of  Calls  Answered  within  120  Seconds 

9  MCSC  Phones  all  lines  busy 

1 0  Preventable  Admission  Rates  for  Prime  Enrollees 

1 1  Direct  Care  Market  Share  -  Outpatient 

12  Direct  Care  Market  Share  -  Inpatient 

1 3  Emergency  Room  Visits/1 000 


1  Satisfaction  with  Quality  of  Health  Care 

2  Overall  Satisfaction  with  Clinic  Visit 

3  JCAHO  Grid  Scores 

4  Malpractice  Claims  Filed 

5  Malpractice  Claims  Paid 

6  Enrollment  at  MTF 

7  Beneficiary  Grievances 

8  Beneficiary  Appeals 

9  Seen  by  PCM 

1 0  Percent  Beneficiary  Satisfied  with  Customer  Service 


1  Bed  days  per  1 000 

2  RWPs  per  1000 

3  Dispositions  per  1000 

4  Claims  Processed  within  30  Days 

5  Cost  per  User  (Cost  per  Enrollee) 

6  Pharmacy  Cost 

7  Cost  per  Bedday 

8  Cost  per  RWP 

9  Nurse  FTE/Weighted  Bed  Day 

1 0  Visits  per  Provider  per  Day 

1 1  Cost  per  Visit  Weighted 

1 2  Visits  per  1 000  Weighted 

13  Radiology  Costs 

14  Laboratory  Costs 

1 5  Capital  Costs 


1  Temporary  Medical  Profiles 

2  Dental  Readiness  (Class  1  and  2) 

3  Dental  Wellness  (Class  1) 

4  Medical  Readiness  T rained  and  Certified 


Standard  Metric  Set 

Appendix  C 

Presented  by  Tenent 

Tenent 

Performance  Measure 

V.  Health  Outcomes 

1 

Health  Status 

2 

Preventive  Care  Measures 

VI.  Special  Interest 

1 

SIDR  Completion 

2 

SADR  Completion 

3 

MEPRS  Completion 

List  of  Facilities  for  Selection  Appendix  D 
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MEMORANDUM  FOR  COMMANDER,  MILITARY  TREATMENT  FACILITY 
SUBJECT:  Standard  Metric  Set 


The  TMA  is  developing  a  standard  set  of  performance  measures  that  will  help  the 
MHS  assess  how  well  we  are  meeting  the  healthcare  needs  of  our  military  beneficiaries. 
These  measures  will  promote  a  more  uniform  and  consistent  way  of  providing  feedback 
to  healthcare  leaders  at  all  levels  of  the  Military  Health  System. 

The  attached  questionnaire  is  designed  to  solicit  your  opinions  about  how  metric 
sets  are  viewed  and  used  within  your  organization.  Your  opinions  will  help  us 
understand  what  features  of  a  standard  set  are  most  likely  to  meet  your  needs  for 
feedback  on  healthcare  service  delivery  performance. 

Please  review  the  attached  information  paper,  complete  the  questionnaire 
provided,  and  return  in  the  enclosed  envelope  by  January  26,  2001.  I  also  ask  that  you 
forward  copies  of  the  attached  questionnaire  to  your  deputy  for  clinical  services  (senior 
physician),  deputy  for  administration  (senior  administrator),  and  chief  nurse  and  ask  that 
they  also  complete  this  survey  and  return  NLT  January  26,  2001. 

1  thank  you  for  your  consideration  of  this  survey.  Please  contact  Major  Alejandro 
Lopez-Duke  at  (703)  681-1730  for  any  questions  regarding  this  survey. 


H.  James  T.  Sears,  M.D. 
Executive  Director 


Attachments: 

1.  Questionnaire 

2.  Information  paper  re.  Standard  Metric  Sets 
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Appendix  G 

An  Information  Paver  for  the  Metric  Set  Questionnaire  Attachment  A 

BACKGROUND  INFORMATION 

In  the  last  five  years,  the  numbers  of  metric  or  performance  measurement  sets  have  increased  throughout 
the  MHS.  The  Services,  Health  Affairs,  and  the  TRICARE  Management  Activity  (TMA)  have  created  a  variety 
of  performance  measurement  sets  to  assist  the  regions  and  the  Military  Treatment  Facilities  (MTFs)  in 
measuring  everything  from  population  health  to  financial  viability.  In  an  effort  to  streamline  the  process  and 
possibly  reduce  duplication  of  effort,  TMA  is  creating  a  standard  metric  set  based  on  the  following  healthcare 
tenets,  access,  quality,  and  cost  to  include  readiness  and  healthcare  outcomes. 

OBJECTIVES 

This  questionnaire  is  submitted  for  your  consideration.  The  questionnaire  is  part  of  a  Graduate 
Management  Project  (GMP)  conducted  by  the  resident,  U.S.  Army-Baylor  University  Masters  Degree  Program 
in  Healthcare  Administration.  The  responses  will  be  kept  confidential.  They  will  be  compiled  and  forwarded  to 
the  TMA  Executive  Director,  Deputy  Director,  Chief  of  Staff,  and  Health  Program  Analysis  and  Evaluation. 
Three  main  goals  to  consider:  1)  to  elicit  input  from  the  MHS  executive  leadership,  TRICARE  regions,  and  the 
MTFs;  2)  to  compare  differences  in  metric  selection  among  services,  facilities,  regions,  and  healthcare 
specialties;  and  3)  to  determine  if  any  major  gaps  exist  between  this  preliminary  standard  metric  set  and 
selected  private  sector  organizations.  • 

RESPONDENTS 

1)  MHS  executive  leadership 

2)  TRICARE  Region  Directors,  Deputy  Directors,  and  Operations  Directors. 

3)  MTF  command  teams:  Commander,  Deputy  Commander  for  Nursing  (Chief  Nurse),  Deputy 
Commander  for  Clinical  Services  (Chief  Physician)  or  Chief  of  Medical  Staff,  Deputy  Commander  for 
Administration  (Chief  Administrator). 

METHODS 

All  data  will  be  collected  using  a  questionnaire.  You  are  not  required  to  travel,  or  prepare  in  advance. 
This  questionnaire  has  specific  aspects  to  it:  1)  all  responses  are  anonymous;  2)  responses  under  the  Likert  scale 
will  be  compiled  and  presented  in  general  terms;  3)  open  ended  responses  will  be  assembled  under  broad 
subject  headings  and  recommendations  will  be  presented  anonimously;  and  4)  demographic  data  are  included  to 
conduct  statistical  analysis  based  on  title,  rank,  service,  profession,  and  organization. 

HOW  LONG  WILL  IT  TAKE? 

This  is  dependent  on  the  amount  of  time  you  invest  in  answering  the  questions.  The  first  and  second 
parts  (Likert  scale)  should  take  no  longer  than  30  minutes.  The  second  section  may  take  up  to  10  minutes 
contingent  on  the  amount  of  input  you  want  to  provide.  A  complete  questionnaire  is  highly  desired  to  insure 
TMA  receives  the  best  possible  responses  from  the  “field”. 

PERSONAL  UTILITY  OF  RESULTS 

By  participating  in  this  questionnaire  you  will  play  a  part  in  assessing  TMA’s  preliminary  standard 
metric  set.  In  addition,  your  comments  will  be  reviewed  and  possibly  incorporated  in  future  performance 
measurement  set  implementation. 

FOR  WHAT  WILL  THE  RESULTS  BE  USED? 

Participation  is  voluntary,  though  highly  encouraged.  Compiled  results  will  be  used  to  assess  the 
proposed  standard  set.  In  addition,  the  results  will  be  incorporated  into  the  student’s  GMP.  Your  feedback  is 
instrumental  in  improving  this  metric  set  and  its  utility  for  possible  use  as  a  foundation  for  regional  or  MTF 
level  performance  measures. 


FOR  FUTHER  INFORMATION  CONTACT:  Major  Alejandro  Lopez-Duke,  (703)  681-1730 


Part  II 

The  STANDARD  SET 


Questionnaire 
MTF  Command  Team 


I.  Purpose  part  II:  To  determine  if  the  foliowing  perfomance  indicators  are  applicable  at 

the  MTF  command  team/executive  level 

Applicability  is  defined  as,  ”is  the  measure  useful  for  decision  making  at  your  level  in  the  MTF?". 

For  example:  If  the  results  of  a  specific  performance  measure  are  pertinent  at  your  management  level, 
rate  it  higher  on  the  scale  from  1  to  7  than  one  that  is  not. 

Set  is  defined  as,  "a  collection  of  performance  measures  or  indicators". 

II.  Instructions:  Please  rate  how  applicable  each  performance  measure  is  for  the  MTF  leadership 

Possible  consideration  for  evaluation  include: 

1)  Does  the  measure  support  management  objectives  based  on  MHS  and  Service 
strategic  plans,  optimization  plans,  or  other  key  management  goals 

2)  Does  the  measure  enhance  (adequate  information)  decision  making  at  your  level 

3)  Will  the  measure  allow  you  to  make  an  informed  decision 

4)  Do  you  consider  the  information  provided  by  each  measure  important  to 
manage  your  organization  more  effectively  and  efficiently 

III.  Please  annotate  demographic  information  (circle  choices): 

Speciaitv/Profession:  Physician/Dentist  Nurse  Administrator  Other 

Service:  Army  Navy  Air  Force  Civilian 


Rank: 

Job  Title:  _ _ 

IV.  Question:  Please  rate  each  measure  on  the  scale  from  least  applicable  to  most  applicable 
for  use  at  the  MTF  command/executive  level 


Please  circle  the  number 

Sample  Scale:  1 - 2 . 3 . 4 . 5 . 6 . 7 

Least  Applicable  Most  Applicable 


Performance  Measures 

Bed  days  per  1000  Enrollees 
Beneficiary  Appeals  per  1 000  Claims 
Pf  :  Beneficiary  Grievances  per  1000  Enrollees 

Capital  Costs  per  Mernber  per  Month _ 

W  Claims  Processed  within  30  Days 
Cost  per Bedday . 

: :  Cost  per  RWP  (Relative  Weighted  Product) 

Cost  per  Enrollee 
Cost  per  Visit  Weighted 
%  Beneficiary  Satisfied  with  Cuslorner  Seryipe 
^  ,  Dental  Readiness  (%  Class  1  and  2) 

Dental  Wellness  (%  Class  1 ) 

Direct  Care  Market  Share  -  Inpatient  workload 


Not  sure 

1 - 2- — 3- — 4 — -5- — 6--7  0 

_  . . .  o_, 

1. — 2 - 3- — 4--5- — 6---7.  0 

1- — 2--3---4 - 5- — 6 . 7  0 

1.. ....2- — 3- — 4 - 5 — -6---7  0 

^  — ..2— •’-—3 — — --4— ■5"""~"6"**"’7  0 

1 - 2- - 3---4 - 5- - 6---7  .'  ■'  0 

■|  ......2- — 3"—'"4 — —5”— -6*— "7  0 

i - 2- — 3- — 4 - 5--6 - 7  -  0 

. .  P 

1 - 2-™-3--4-™5- — 6---7  '0 

1. - 0 

1.. ...-2— — 3 - 4— — 5— .!:6— — 7  0 
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Part  II 

The  STANDARD  SET 


Questionnaire 
MTF  Command  Team 


Sample  Scale;  1 - 2 - 3 . 4 - 5 - 6 - 7 

Least  Applicable  Most  Applicable  Not  sure 


Direct  Care  Market  Share  -  Outpatient  workload 

Dispositions  per  10p0  Enrollees 

Ease  of  Making  Appointment  by  Phone 

Emergency  Room  Visits  per  i  000  Enrollees 

%  Eligibles  Enrolled  to  MTF 

Health  Status  (self  reported  on  SF  12--mean  score) 

JCAHO  Grid  Scores 

Laboratory  Costs  per  , Mernbdr  per  Month 

Malpractice  Claims  Failed  per  100  Physician 

Malpractice  Claims  Paid  per  1 00  Physicians  f 

MCSC  Percent  .of  Calls  Answered  within  2  minutes 

MCSC  Phones -“  %  of  all  lines  busy 

%  Providers  Medical  Readiness  Trairied  and  Certified 

MEPRS  Completion  Rate 

Nurse  FTE  per  Weighted  Bed  Day 

Overall  Satisfaction  with  Clinic  Visit 

Patient  Satisfaction  with  Access 

Pitieht  Satisfaction  with  Waiting  Times  '  ^ 

Pharmacy  Cost  per  Member  per  Month 
Preventable  Admission  Rates  for  Prime  Enrollees 
Preventive  Care  Measures  (ex.  Pap  smears,  prostate) 
Radiology  Costs  per  Member  per  Month 

RWP?_p®L.t99P,®nr9P®®?.  . . . . 

SADR  Completion  Rate 

Satisfaction  with  Aiccess  to  Specialist  _ 

Satisfaction  with  Access  to  Healthcare 
Satisfaction  with  Quality  of  Health  Care 
%  of  Enrollees  Seen  by  their  PCM 
SI  DR  Completion  Rate 

%  of  AD  Members  with  Temporary  Medical  Profiles 

Visits  per  1000  VVeighted 

Visits  per  Provider  per  Day 

Wait  Time  at  Appointment  (30  minute  standard) 

WaitTime  for  Appointment  (TRICARE  standards) 


Standard  Set 


2 


Part  III 


Questionnaire 
MTF  Command  Team 


i.  Purpose  part  III:  To  determine  if  there  are  any  additonal  performance  measures/metrics 
that  should  be  included  in  a  standard  metric  set 

11.  Instructions:  Please  list  any  performance  measures  you  think  should  be  included  in 
a  MHS  wide  standard  metric  set 

Possible  criteria  for  selection: 

1)  does  the  measure  support  management  objectives  based  on  MHS  and  service  specific 
strategic  plans,  optimization  plans,  or  other  key  management  goals, 

2)  does  the  measure  enhance  (adequate  Information)  decision  making  at  your  level, 

3)  will  the  measure  allow  you  to  make  an  informed  decision, 

4)  do  you  consider  the  information  provided  by  the  measure  important  to 
enable  you  to  manage  your  organization  rrtore  effectively  and  efficiently,  and 

5)  Is  data  readily  available  to  build  recommended  performance  measures/indicators 


III.  Question: 

Are  there  other  metrics  or  performance  measures/indicators  that  were  not  reflected  on  the  standard 
set  in  part  II? 

If  so,  please  list 


Open  Ended  Question 
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Part  I  Appendix  I 

Executive  Level  Questionnaire 

I.  Purpose  of  questionnaire:  To  determine  perfomance  indicators  for  an  executive  level  metric  set 

Applicability  is  defined  as:  "is  the  measure  useful  for  decision  making  at  your  level  in  the  MHS?". 
For  example:  If  the  results  of  a  specific  performance  measure  are  pertinent  at  the  executive  level,  rate  it 
higher  on  the  scale  from  1  to  7  than  one  that  is  not. 

II.  Instructions:  Rate  how  applicable  each  performance  measure  is  for  inclusion  into  an  executive 

level  metric  set. 

Possible  considerations  for  evaluation  include: 

A)  Does  the  measure  support  management  objectives  based  on  MHS  and  Service  strategic  plans, 
optimization  plans,  or  other  key  corporate  goals, 

B)  Does  the  measure  enhance  (adequate  information)  decision  making  at  the  executive  level. 

C)  Will  the  measure  allow  the  executive  to  make  an  informed  decision, 

D)  Does  the  executive  consider  the  information  provided  by  each  measure  important  to 
make  strategic  level  enterprise  wide  decisions. 

III.  Please  annotate  demographic  information  (circle  choices): 


Soecialtv/Profession: 

Physician/Dentist 

Nurse 

Administrator  Other 

Service: 

Army 

Navy 

Air  Force  Civilian 

Rank: 

Job  Title: 

IV.  Question:  Please  rate  each  measure  on  the  scale  from  least  applicable  to  most  applicable 
for  inclusion  in  an  executive  level  metric  set 


Please  circle  the  number 

Sample  Scale:  1 - 2 - 3 - 4 - 5 - 6 - 7 

Least  Applicable  Most  Applicable 


Performance  Measures 

Not  sure 

I  . Bed  days  per  1 000  Enrollees  1 - 2 - 3 - 4 - 5 - 6 - 7  0 

Beneficiary  Appeals  per  1 000  Claims  1  -- — 2- — --3— —4— —5— —6— —7  0 

I'  -  '  Beneficiary  Grievances  per  1660  Enrollees  0 

Capital  Costs  per  Member  per  Month  ,  0 

f.  j  Claims  Processed  within  30  Days  1 - 2 - 3 - 4 - 5 - 6 — --7  •  .  ;  i  0 

Cost  per  Bedday . . . .  _  __  _ .  1  "-3— -"4----"5 — .  ,0 

f  Cost  per  RWP  (Relative  Weighted  Product)  1 - 2— ---3 - 4 - 5 - 6 - 7  '  •  ,  ,0 

Cost  per  Enrollee  1 - 2 - 3 . 4--- — 5-"— 6-- — -7  0 

I’  Cost  per  Visit  Weighted  1 - 2 - 3 - 4 - 5 — -6 - 7  0 

%  Beneficiary  Satisfied  with  Customer  Service  1------2------3------4------5------6— ---7  ,  0 

I  Dental  Readiness  (%  Class  1  and  2)  1 - 2 - 3 - 4 - 5 - 6 - 7  0 

Dental  Vyell_ness  (®/o  Class  1)  . . . . . — -7  0 

i  Direct  Care  Market  Share  -  Inpatient  workload  1 - 2 - 3 - 4 - 5------6 - 7  '  '  .  0 


Standard  Metric  Set 


1 


Part  I 


Appendix  I 


Executive  Level  Questionnaire 

Sample  Scale:  1 . 2 . 3 . 4 . 5 . 6 . 7 

Least  Applicable  Most  Applicable  Not  sure 


r 

r" 

1/ 


r"'”' 


I 


I 


I 

1. 

f- 


Direct  Care  Market  Share  -  Outpatient  workload . . . 

Dispositions  per  1 000  Enrollees  .1 - 2 — --3 - 4 5 6 - 7 

Ease  of  Making  Appointment  by  Phpne  1 - 2 - 3— 7“4-- — 5:— 

Emergency  Room  Visits  per  1000  Enrollees  1 - 2 - 3 - 4 - 5 - 6 - 7 

%  Eligibles  Enrolled  to  MTF  1 

Health  Status  (self  reported  on  SF  12--mean  score)  — 3— --4— — 5 - 6 - 7  : 

JCAHO  Grid  Scores  1 - 2 - 3 4;------5-™--6-;--"7  . 

Laboratory  Costs  per  Member  per  Month  :  1 - 2 - 3 - 4 5 6 - 7 

Malpractice  Claims  Filed  per  100  Physicians  ^  . . 1 - 2-— --3 — -6— ---7 

Malpractice  Claims  Paid  per  100  Physicians  1 - 2 - 3 - 4 — ---5 - 6 - 7 

MCSC  Percerit  of  Calls  Ansvyered  within  2  minutes  1 - 2 - 3 - 4— ---5 - 6 . -7 

MCSC  Phones —  %  of  all  iines  busy  1......2.....-3------4------5----"^^^^^  - 

%  Providers  Medical  Readiness  Trained  and  Certified  1 - 2 — -3 - 4— —5— —6— —7 

MEPRS  Completion  Rate  ,  1- — -.2---i-^^3------4-----'5------6--™^ 

Nurse  FTE  per  Weighted  Bed  Day  1 - 2 . 3 - 4 . 5 . 6 . 7 

Ovefall  Satisfaction  with  Clinic  Visit  1 - 2 - 3 - 4 - 5-- — 6 - 7 

Patient  Satisfaction  with  Access  1 . — 2 - 3 - 4- — -5— —6 - 7 

Patient  Satisfaction  with  Waiting  Times  1 - 2 - 3 - 4 - 5 - 6 - 7 

Pharmacy  Cost  per  Mernber  per  Mo  1 - 2 - 3 4.— -5— —6— -—7 

Preventable  Admission  Rates  for  Prime  Enrollees  1 - 2 - 3 - 4 - 5 - 6 - 7 

Preventive  Care  Measures  (ex.  Pap  smears,  prostate)  1 - 2 - 3 - 4 - 5 - 6 - 7 

Radiology  Costs  per  Member  per  Month  1 - 2 - 3 - 4 - 5 - 6 - 7 

RWPs  per  1 000  enrollees  1 - 2-- — 3 - 4 — "-7  _ 

SADR  Completion  Rate  1 - 2 — "3 - 4 - 5 - 6 - 7 

Satisfaction  with  Access  to  Specialist  _ _  1-..— 2 - 3 4— —5 6 - 7 

Satisfaction  with  Access  to  Healthcare  1 - 2 - 3 - 4 - 5 - 6 - 7 

Satisfaction  vyith  Quality  of  Health  Care  1  — -2 - 3 - 4 — .5... ..-6— —7 

%  of  Enrollees  Seen  by  their  PGM  ;  1 — --2 — —3 — --4— ---5 - 6 - 7 

SIDR  Completion  Rate  1 - 2 - 3— —4 5 6- — 7 

%  of  AD  Members  with  temporary  Medical  Profiles  ^  1-r- — 2- — -3 - 4 — —5 — ^6 - 7 

Visits  per  1000  Weighted  1 ------2------3------4------5^ 

Visits  per  Provider  per  Day  1 — -2----,--3------4------5------6------7^  ^ ; 

Wait  Time  at  Appointment  (30  minute  standard)  1 - 2 - 3— — 4-— -s-.-.-.g— — 7 

’Wait  Time  for  Appointment  (TRICARE  standards)  .1 - 2 - 3 - 4 - 5 - 6 - 7 


Standar(J  Metric  Set 
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Part  II 


Questionnaire 
MHS  Executives 


I.  Purpose  part  II:  To  determine  if  there  are  any  additonal  performance  measures/metrics 

that  should  be  included  in  a  standard  metric  set 

II.  Instructions:  Please  list  any  performance  measures  you  think  should  be  included  in 

a  MHS  wide  standard  metric  set 

Possible  criteria  for  selection: 

1)  does  the  measure  support  management  objectives  based  on  MHS  and  service  specific 
strategic  plans,  optimization  plans,  or  other  key  management  goals, 

2)  does  the  measure  enhance  (adequate  Information)  decision  making  at  your  level, 

3)  will  the  measure  allow  you  to  make  an  informed  decision, 

4)  do  you  consider  the  information  provided  by  the  measure  important  to 
enable  you  to  manage  your  organization  more  effectively  and  efficiently,  and 

6)  is  data  readily  available  to  build  recommended  performance  measures/indicators 


III.  Question: 

Are  there  other  metrics  or  performance  measures/indicators  that  were  not  reflected  on  the  standard 
set  in  part  I? 

If  so,  please  list 


Open  Ended  Question 


1 


Appendix  J 

Operational  Definition  of  Variables 
Defined  as: 

Army,  Navy,  Air  Force 
Physician,  Administrator,  Nurse 
Executive,  Region,  MTF 


Defined  as: 

See  appendix  C 
Topic  areas  follow: 


1 .  Services  should  review  existing  sets 

2.  Too  many  sets  in  MHS 

3.  I  am  aware  of  corporate  management  objectives  in  MHS  strategic  plan 

4.  I  am  aware  of  service  specific  strategic  goal 

5.  Sets  are  tied  (mapped)  to  management  objectives 

6.  Performance  measures  help  organizations  achieve  strategic  goals 

7.  The  standard  metric  set  support  enterprise  wide  strategic  goals 

8.  The  standard  metric  set  support  service  specific  strategic  goals 

9.  Metric  sets  are  best  prepared  under  a  single  proponent 

10.  Standard  metric  set  should  be  implemented  in  the  MHS 

11.  Standard  methodology  to  calculate  performance  measures  should  exist 

12.  The  standard  metric  set  should  be  the  template  for  other  service  sets 

13.  The  standard  metric  set  represents  a  balance  approach  to  metric  selection 


Metric  Sets  are  currently  mapped  back  to 
Management  Objectives 

Univariate  Anaiysis  of  Variance 


Between-Subjects  Factors 


N 

SERVICE  1 

33 

2 

27 

3 

44 

LEVELCMD  1 

89 

2 

15 

PROFESN  1 

34 

2 

29 

'  3 

41 

Appendix  K 


Descriptive  Statistics 


Tests  of  Between-Subjects  Effects 


Dependent  Variable:  MAPPED 


Source 

df 

F 

mm 

Corrected  Model 

43.183® 

14 

3.085 

1.931 

.033 

Intercept 

442.883 

1 

442.883 

277.264 

.000 

SERVICE 

8.263 

2 

4.132 

2.587 

.081 

LEVELCMD 

2.171 

1 

2.171 

1.359 

.247 

PROFESN 

9.555 

2 

4.778 

2.991 

.055 

SERVICE  *  LEVELCMD 

4.187 

2 

2.094 

1.311 

.275 

SERVICE  *  PROFESN 

3.523 

4 

.881 

.551 

.698 

LEVELCMD  *  PROFESN 

13.700 

2 

6.850 

4.288 

.017 

SERVICE  ‘  LEVELCMD 
*  PROFESN 

1.117 

1 

1.117 

.699 

.405 

Error 

142.163 

89 

1.597 

Total 

2098.000 

104 

Corrected  Total 

185.346 

103 

a.  R  Squared  =  .233  (Adjusted  R  Squared  =  .112) 


Profile  Plots 


PROFESN  *  LEVELCMD  *  SERVICE 


Estimated  Marginal  Means  of  MAPPED 


At  SERVICE  =  1 


PROFESN 


Estimated  Marginal  Means  Estimated  Marginal  Means 


Estimated  Marginal  Means  of  MAPPED 
At  SERVICE  =  2 


PROFESN 

Non-estimable  means  are  not  plotted 


Estimated  Marginal  Means  of  MAPPED 
At  SERVICE  =  3 


LEVELCMD 


1  2  3 

PROFESN 


Non-estimable  means  are  not  plotted 


Metric  Sets  are  Currently  Mapped  back 
to  Management  Objectives 


Tukeys  HSD 
Appendix  L 


Univariate  Analysis  of  Variance 


Warnings 


Post  hoc  tests  are  not  performed  for 
LEVELCMD  because  there  are 
fewer  than  three  groups. 


Between-Subjects  Factors 


N 

SERVICE  1 

33 

2 

27 

3 

44 

LEVELCMD  1 

89 

2 

15 

PROFESN  1 

34 

2 

29 

3 

41 

Tests  of  Between-Subjects  Effects 


Dependent  Variable:  MAPPED 


Source 

df 

F 

mm 

Corrected  Model 

43.183® 

14 

3.085 

1.931 

.033 

Intercept 

442.883 

1 

442.883 

277.264 

.000 

SERVICE 

8.263 

2 

4.132 

2.587 

.081 

LEVELCMD 

2.171 

1 

2.171 

1.359 

.247 

PROFESN 

9.555 

2 

4.778 

2.991 

.055 

SERVICE  *  LEVELCMD 

4.187 

2 

2.094 

1.311 

.275 

SERVICE  *  PROFESN 

3.523 

4 

.881 

.551 

.698 

LEVELCMD  *  PROFESN 

13.700 

2 

6.850 

4.288 

.017 

SERVICE  *  LEVELCMD 
*  PROFESN 

1.117 

1 

1.117 

.699 

.405 

Error 

142.163 

89 

1.597 

Total 

2098.000 

104 

Corrected  Total 

185.346 

103 

a.  R  Squared  =  .233  (Adjusted  R  Squared  =  .112) 


Post  Hoc  Tests 


SERVICE 


Dependent  Variable:  MAPPED 
Tukey  HSD 


Multiple  Comparisons 


(J)  SERVICE 

Mean 

Difference 

Std.  Error 

Sig. 

95%  Confidence  Interval 

Lower  Bound 

Upper  Bound 

1 

2 

-.73 

.33 

.072 

-1.51 

5.1  IE-02 

3 

* 

CO 

.29 

.035 

-1.43 

-4.1  IE-02 

2 

1 

.73 

.33 

.072 

-5.11E-02 

1.51 

3 

-4.21  E-03 

.31 

1.000 

-.74 

.73 

3 

1 

.73* 

.29 

.035 

4.1  IE-02 

1.43 

2 

4.2  IE-03 

.31 

1.000 

-.73 

.74 

Based  on  observed  means. 

*.  The  mean  difference  is  significant  at  the  .05  level. 


Homogeneous  Subsets 


MAPPED 


Tukey  HSD®’'’-'^ 


SERVICE 

N 

Subset 

1 

1 

33 

3.79 

27 

4.52 

ll^H 

44 

4.52 

.051 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  1 .597. 

a.  Uses  Harmonic  Mean  Sample  Size  =  33.308. 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


PROFESN 


Dependent  Variable:  MAPPED 
Tukey  HSD 


Multiple  Comparisons 


(I)  PROFESN 

(J)  PROFESN 

Mean 

Difference 

(l-J) 

Std.  Error 

m 

95%  Confidence  Interval 

Lower  Bound 

1 

2 

3.25E-02 

.32 

.994 

-.73 

.79 

3 

.29 

.29 

.586 

-.41 

.99 

2 

1 

-3.25E-02 

.32 

.994 

-.79 

.73 

3 

.26 

.31 

.680 

-.47 

.99 

3 

1 

-.29 

.41 

2 

-.26 

.47 

Based  on  observed  means. 


Homogeneous  Subsets 


MAPPED 

Tukey  HSD®’’’  '' 


PROFESN 

N 

Subset 

1 

3 

41 

4.12 

2 

29 

4.38 

1 

34 

4.41 

Sig. 

.613 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  1 .597. 

a.  Uses  Harmonic  Mean  Sample  Size  =  33.981 . 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


%  Active  Duty  with  Temporary  Profiles 


Appendix  M 


Univariate  Analysis  of  Variance 


Between-Subjects  Factors 


N 

SERVICE  1 

34 

2 

26 

3 

46 

LEVLCMD  1 

85 

2 

17 

3 

4 

PROFSION  1 

38 

2 

27 

3 

41 

Descriptive  Statistics 


Dependent  Variable:  PROFILES 


SERVICE 

LEVLCMD  PROFSION 

Mean 

Std.  Deviation 

N 

1 

1 

1 

3.80 

2.10 

10 

2 

6.13 

.64 

8 

3 

4.00 

2.06 

9 

Total 

4.56 

2.01 

27 

2 

1 

7.00 

, 

1 

2 

3.00 

, 

1 

3 

3.00 

2.83 

4 

Total 

3.67 

2.73 

6 

3 

1 

. 

1 

Total 

, 

1 

Total 

1 

3.92 

2.19 

12 

2 

5.78 

1.20 

9 

3 

3.69 

2.25 

13 

Total 

4.32 

2.14 

34 

2 

1 

1 

4.57 

1.40 

7 

2 

3.71 

1.98 

7 

3 

3.43 

.98 

7 

Total 

3.90 

1.51 

21 

2 

3 

4.67 

1.15 

3 

Total 

4.67 

1.15 

3 

3 

1 

4.00 

1.41 

2 

Total 

4.00 

1.41 

2 

Total 

1 

4.44 

9 

2 

3.71 

7 

3 

3.80 

1.14 

Total 

4.00 

1.44 

3 

1 

1 

4.33 

1.11 

15 

2 

4.91 

1.81 

11 

3 

4.55 

1.86 

11 

Total 

4.57 

1.56 

37 

2 

1 

^■sa 

1 

3 

1.83 

7 

Total 

i  4.88 

1.73 

8 

3 

1 

, 

1 

Total 

1 

Total 

1 

4.35 

1.06 

17 

2 

4.91 

1.81 

11 

3 

4.72 

1.81 

Total 

4.63 

1.55 

Total 

1 

1 

4.22 

32 

2 

4.96 

26 

3 

4.07 

1.75 

27 

Total 

4.40 

1.71 

85 

2 

1 

5.50 

2.12 

2 

2 

3.00 

. 

1 

3 

4.36 

2.10 

14 

Total 

4.41 

2.03 

17 

3 

1 

1.50 

4 

Total 

1.50 

4 

Total 

1 

4.24 

2 

4.89 

3 

4.17 

1.86 

41 

Total 

4.38 

1.74 

106 

Tests  of  Between-Subjects  Effects 


Dependent  Variable:  PROFILES 


Source 

df 

Li. 

mm 

Corrected  Model 

67.937a 

17 

3.996 

1.401 

.155 

Intercept 

477.048 

1 

477.048 

167.273 

.000 

SERVICE 

1.880 

2 

.940 

.330 

.720 

LEVLCMD 

2 

2.582 

.905 

.408 

PROFSION 

2 

3.013 

1.057 

.352 

SERVICE  *  LEVLCMD 

4 

2.727 

.956 

.436 

SERVICE  *  PROFSION 

4 

7.947 

2.786 

.031 

LEVLCMD  * 

PROFSION 

14.072 

2 

7.036 

2.467 

.091 

SERVICE  *  LEVLCMD  * 
PROFSION 

9.009 

1 

9.009 

3.159 

.079 

Error 

250.969 

88 

2.852 

Total 

2350.000 

106 

Corrected  Total 

318.906 

105 

a.  R  Squared  =  .213  (Adjusted  R  Squared  =  .061) 


Profile  Plots 


PROFSION  *  LEVLCMD  *  SERVICE 


Estimated  Marginal  Means  of  PROFILES 


PROFSION 


Non-estimable  means  are  not  plotted 


Estimated  Marginal  Means  Estimated  Marginal  Means 


Estimated  Marginal  Means  of  PROFILES 
At  SERVICE  =  2 


PROFSION 

Non-estimable  means  are  not  plotted 


Estimated  Marginal  Means  of  PROFILES 


PROFSION 


Non-estimable  means  are  not  plotted 


Relative  Weighted  Product  per 
1000  Enrollees 


Univariate  Analysis  of  Variance 


Between-Subjects  Factors 


N 

SERVICE  1 

25 

2 

24 

3 

33 

LEVLCMD  1 

63 

2 

15 

3 

4 

PROFSION  1 

29 

2 

15 

3 

38 

Appendix  N 


Descriptive  Statistics 


Dependent  Variable:  RWPENROL 


Tests  of  Between-Subjects  Effects 


Dependent  Variable:  RWPENROL 


Source 

■flglllW 

df 

F 

■■ 

Corrected  Model 

50.568® 

16 

3.161 

1.341 

.201 

Intercept 

878.855 

1 

878.855 

372.911 

.000 

SERVICE 

1.267 

2 

.633 

.269 

.765 

LEVLCMD 

9.105 

2 

4.553 

1.932 

.153 

PROFSION 

26.163 

2 

13.081 

5.551 

.006 

SERVICE  *  LEVLCMD 

4.037 

4 

1.009 

.428 

.788 

SERVICE  *  PROFSION 

.926 

4 

.232 

.098 

.983 

LEVLCMD  * 

PROFSION 

3.632 

1 

3.632 

1.541 

.219 

SERVICE  *  LEVLCMD  * 
PROFSION 

1.741 

1 

1.741 

.739 

.393 

Error 

153.188 

65 

2.357 

Total 

2002.000 

82 

Corrected  Total 

203.756 

81 

a.  R  Squared  =  .248  (Adjusted  R  Squared  =  .063) 


Profile  Plots 


PROFSION  *  LEVLCMD  *  SERVICE 


Estimated  Marginal  Means  of  RWPENROL 


At  SERVICE  =  1 


PROFSION 

Non-estimable  means  are  not  plotted 


Estimated  Marginal  Means  Estimated  Marginal  Means 


Estimated  Marginal  Means  of  RWPENROL 
At  SERVICE  =  2 


PROFSION 

Non-estimable  means  are  not  plotted 


Estimated  Marginal  Means  of  RWPENROL 
At  SERVICE  =  3 

6.5 
6.0 

5.5 
5.0 

4.5 
4.0 

3.5 
3.0 

i  2  3 

PROFSION 


Non-estimable  means  are  not  plotted 


MCSC  Phones,  %  of  all  Lines  Busy 

Univariate  Analysis  of  Variance 


Between-Subjects  Factors 


N 

SERVICE  1 

32 

2 

28 

3 

43 

LEVLCMD  1 

83 

2 

16 

3 

4 

PROFSION  1 

38 

2 

25 

3 

40 

Appendix  0 


Descriptive  Statistics 


Dependent  Variable:  MCSPHONE 


SERVICE 

LEVLCMD 

PROFSION 

Mean 

Std.  Deviation 

N 

1 

1 

1 

5.00 

10 

2 

5.29 

7 

3 

5.38 

.74 

8 

Total 

5.20 

1.38 

25 

2 

1 

6.00 

. 

1 

2 

4.00 

. 

1 

3 

4.75 

1.50 

4 

Total 

4.83 

1.33 

6 

3 

1 

4.00 

, 

1 

Total 

4.00 

1 

Total 

1 

1.54 

12 

2 

1.64 

8 

3 

1.03 

12 

Total 

5.09 

1.35 

32 

2 

1 

1 

5.38 

1.41 

8 

2 

4.14 

1.95 

7 

3 

4.63 

1.19 

8 

Total 

4.74 

1.54 

23 

2 

3 

6.00 

1.00 

3 

Total 

6.00 

1.00 

3 

3 

1 

2.00 

1.41 

2 

Total 

2.00 

1.41 

2 

Total 

1 

4.70 

1.95 

10 

2 

4.14 

1.95 

3 

5.00 

1.26 

Total 

4.68 

1.68 

3 

1 

1 

4.07 

1.38 

14 

2 

5.20 

1.48 

10 

3 

4.73 

1.49 

11 

Total 

4.60 

1.48 

35 

2 

1 

6.00 

1 

3 

5.00 

1.79 

6 

Total 

5.14 

1.68 

7 

3 

1 

6.00 

1 

Total 

1 

Total 

1 

4.31 

1.45 

16 

2 

5.20 

1.48 

10 

3 

4.82 

1.55 

17 

Total 

4.72 

1.50 

43 

Total 

1 

1 

4.69 

32 

2 

4.92 

24 

3 

4.89 

1.22 

27 

Total 

4.82 

1.47 

83 

2 

1 

6.00 

.00 

2 

2 

1 

3 

5.15 

1.52 

13 

Total 

5.19 

1.42 

16 

3 

1 

HBI 

2.08 

4 

Total 

2.08 

4 

Total 

1 

4.63 

2 

4.88 

3 

4.97 

1.31 

40 

Total 

4.83 

1.50 

103 

Tests  of  Between-Subjects  Effects 

Dependent  Variable:  MCSPHONE 


Source 

df 

F 

■■ 

Corrected  Model 

45.483® 

17 

2.675 

1.227 

.262 

Intercept 

632.991 

1 

632.991 

290.251 

.000 

SERVICE 

5.236 

2 

2.618 

1.201 

.306 

LEVLCMD 

4.349 

2 

2.174 

.997 

.373 

PROFSION 

2.872 

2 

1.436 

.658 

.520 

SERVICE  *  LEVLCMD 

21.738 

4 

5.435 

2.492 

.049 

SERVICE  *  PROFSION 

12.908 

4 

3.227 

1.480 

.216 

LEVLCMD  * 

PROFSION 

4.590 

2 

2.295 

1.052 

.354 

SERVICE  *  LEVLCMD  * 
PROFSION 

3.393E-04 

1 

3.393E-04 

.000 

.990 

Error 

185.371 

2.181 

Total 

2629.000 

Corrected  Total 

230.854 

102 

a.  R  Squared  =  .197  (Adjusted  R  Squared  =  .036) 


Profile  Plots 


PROFSION  *  LEVLCMD  *  SERVICE 


Estimated  Marginal  Means  of  MCSPHONE 


At  SERVICE  =  1 


LEVLCMD 


PROFSION 

Non-estimable  means  are  not  plotted 


Estimated  Marginal  Means  Estimated  Marginal  Means 


Estimated  Marginal  Means  of  MCSPHONE 
At  SERVICE  =  2 


PROFSION 

Non-estimable  means  are  not  plotted 


Estimated  Marginal  Means  of  MCSPHONE 


PROFSION 


Non-estimable  means  are  not  plotted 


Malpractice  Claims  Filed  per  100  Physicians 


Appendix  P 


Univariate  Anaiysis  of  Variance 


Between-Subjects  Factors 


N 

SERVICE  1 

34 

2 

31 

3 

46 

LEVLCMD  1 

89 

2 

17 

3 

5 

PROFSION  1 

40 

2 

28 

3 

43 

Descriptive  Statistics 

Dependent  Variable:  MALPRCFL 


Tests  of  Between-Subjects  Effects 


Dependent  Variable:  MALPRCFL 


Source 

df 

F 

warn 

Corrected  Model 

78.502® 

17 

4.618 

1.566 

.090 

Intercept 

450.642 

1 

450.642 

152.803 

.000 

SERVICE 

.692 

2 

.346 

.117 

.889 

LEVLCMD 

6.388 

2 

3.194 

1.083 

.343 

PROFSION 

10.798 

2 

5.399 

1.831 

.166 

SERVICE  *  LEVLCMD 

.163 

4 

4.078E-02 

.014 

1.000 

SERVICE  *  PROFSION 

34.716 

4 

8.679 

2.943 

.024 

LEVLCMD  * 

PROFSION 

5.326 

2 

2.663 

.903 

.409 

SERVICE  *  LEVLCMD  * 
PROFSION 

.162 

1 

.162 

.055 

.815 

Error 

274.273 

93 

2.949 

Total 

2169.000 

111 

Corrected  Total 

352.775 

110 

a.  R  Squared  =  .223  (Adjusted  R  Squared  =  .080) 


Profile  Plots 


PROFSION  *  LEVLCMD  *  SERVICE 


Estimated  Marginal  Means  of  MALPRCFL 


At  SERVICE  =  1 


PROFSION 

Non-estimable  means  are  not  plotted 


Estimated  Marginal  Means  Estimated  Marginal  Means 


Estimated  Marginal  Means  of  MALPRCFL 
At  SERVICE  =  2 


PROFSION 

Non-estimable  means  are  not  plotted 


Estimated  Marginal  Means  of  MALPRCFL 


At  SERVICE  =  3 


PROFSION 


Non-estimable  means  are  not  plotted 


Dental  Wellness 


Appendix  Q 


Univariate  Anaiysis  of  Variance 


Between-Subjects  Factors 


N 

SERVICE  1 

32 

2 

30 

3 

47 

LEVLCMD  1 

87 

2 

17 

3 

5 

PROFSION  1 

40 

2 

28 

3 

41 

Descriptive  Statistics 
Dependent  Variable:  DENWELL 


Tests  of  Between-Subjects  Effects 


Dependent  Variable;  DENWELL 


Source 

df 

F 

mm 

Corrected  Model 

81.067^ 

17 

4.769 

2.279 

.007 

Intercept 

673.340 

1 

673.340 

321.846 

.000 

SERVICE 

.626 

2 

.313 

.150 

.861 

LEVLCMD 

5.200 

2 

2.600 

1.243 

.293 

PROFSION 

2 

1.922 

.919 

.403 

SERVICE  *  LEVLCMD 

4 

4.605 

2.201 

.075 

SERVICE  *  PROFSION 

4 

6.087 

2.909 

.026 

LEVLCMD  * 

PROFSION 

.170 

2 

8.503E-02 

.041 

.960 

SERVICE  *  LEVLCMD  * 
PROFSION 

13.672 

1 

13.672 

6.535 

.012 

Error 

190.383 

2.092 

Total 

3128.000 

Corrected  Total 

271.450 

108 

a.  R  Squared  =  .299  (Adjusted  R  Squared  =  .168) 


Profile  Plots 


PROFSION  *  LEVLCMD  *  SERVICE 


Estimated  Marginal  Means  of  DENWELL 


PROFSION 


Non-estimable  means  are  not  plotted 


Estimated  Marginal  Means  Estimated  Marginal  Means 


Estimated  Marginal  Means  of  DENWELL 
At  SERVICE  =  2 


PROFSION 

Non-estimable  means  are  not  plotted 


Estimated  Marginal  Means  of  DENWELL 


At  SERVICE  =  3 


PROFSION 


Non-estimable  means  are  not  plotted 


Dental  Readiness 


Appendix  R 


Univariate  Analysis  of  Variance 


Between-Subjects  Factors 


N 

SERVICE  1 

32 

2 

30 

3 

47 

LEVLCMD  1 

87 

2 

17 

3 

5 

PROFSION  1 

40 

2 

28 

3 

41 

Descriptive  Statistics 


Dependent  Variable;  DENREAD 


Tests  of  Between-Subjects  Effects 


Dependent  Variable:  DENREAD 


Source 

df 

F 

mm 

Corrected  Model 

75.062^ 

17 

4.415 

2.573 

.002 

Intercept 

817.370 

1 

817.370 

476.234 

.000 

SERVICE 

1.531 

2 

.765 

.446 

.642 

LEVLCMD 

9.391 

2 

4.696 

2.736 

.070 

PROFSION 

.497 

2 

.249 

.145 

.865 

SERVICE  *  LEVLCMD 

16.550 

4 

4.138 

2.411 

.055 

SERVICE  *  PROFSION 

18.763 

4 

4.691 

2.733 

.034 

LEVLCMD  * 

PROFSION 

.222 

2 

.111 

.065 

.937 

SERVICE  *  LEVLCMD  * 
PROFSION 

8.872 

1 

8.872 

5.169 

.025 

Error 

156.185 

1.716 

Total 

3534.000 

Corrected  Total 

231.248 

108 

a.  R  Squared  =  .325  (Adjusted  R  Squared  =  .198) 


Profile  Plots 


PROFSION  *  LEVLCMD  *  SERVICE 


Estimated  Marginal  Means  of  DENREAD 


At  SERVICE  =  1 


PROFSION 

Non-estimable  means  are  not  plotted 


Estimated  Marginal  Means  Estimated  Marginal  Means 


Estimated  Marginal  Means  of  DENREAD 
At  SERVICE  =  2 


PROFSION 

Non-estimable  means  are  not  plotted 


Estimated  Marginal  Means  of  DENREAD 


At  SERVICE  =  3 


LEVLCMD 


PROFSION 


Non-estimable  means  are  not  plotted 


%  Active  Duty  with  Temporary  Profiles  Appendix  S 

Tukey ' s  HSD 

Univariate  Analysis  of  Variance 


Between-Subjects  Factors 


N 

SERVICE  1 

34 

2 

26 

3 

46 

LEVLCMD  1 

85 

2 

17 

3 

4 

PROFSION  1 

38 

2 

27 

3 

41 

Tests  of  Between-Subjects  Effects 

Dependent  Variable:  PROFILES 


Source 

df 

F 

mm 

Corrected  Model 

67.937^ 

17 

3.996 

1.401 

.155 

Intercept 

477.048 

1 

477.048 

167.273 

.000 

SERVICE 

1.880 

2 

.940 

.330 

.720 

LEVLCMD 

5.163 

2 

2.582 

.905 

.408 

PROFSION 

6.026 

2 

3.013 

1.057 

.352 

SERVICE  *  LEVLCMD 

10.909 

4 

2.727 

.956 

.436 

SERVICE  *  PROFSION 

31.787 

4 

7.947 

2.786 

.031 

LEVLCMD  • 

PROFSION 

14.072 

2 

7.036 

2.467 

.091 

SERVICE  *  LEVLCMD  * 
PROFSION 

9.009 

1 

9.009 

3.159 

.079 

Error 

250.969 

88 

2.852 

Total 

2350.000 

106 

Corrected  Total 

318.906 

105 

a.  R  Squared  =  .213  (Adjusted  R  Squared  =  .061) 


Post  Hoc  Tests 


SERVICE 


Dependent  Variable:  PROFILES 
Tukey  HSD 


Multiple  Comparisons 


Mean 

Difference 

(l-J) 

Std.  Error 

m 

95%  Confidence  Interval 

Lower  Bound 

KI1t!S!31Z!SITIifA 

1 

2 

.32 

.44 

-.73 

3 

-.31 

.38 

-1.22 

■■PI 

2 

1 

-.32 

.44 

-1.37 

.73 

3 

-.63 

.41 

-1.62 

.36 

3 

1 

.38 

.702 

-.60 

2 

.41 

.286 

-.36 

■■BI3 

Based  on  observed  means. 


Homogeneous  Subsets 


PROFILES 

Tukey  HSD®'**'® 


SERVICE 

N 

Subset 

1 

2 

26 

1 

34 

46 

.283  1 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  2.852. 

a.  Uses  Harmonic  Mean  Sample  Size  =  33.477. 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


PROFSION 


Dependent  Variable:  PROFILES 
Tukey  HSD 


Multiple  Comparisons 


(1)  PROFSION 

(J)  PROFSION 

Mean 

Difference 

(l-J) 

Std.  Error 

Sig. 

95%  Confidence  Intervai 

Lower  Bound 

Upper  Bound 

1 

2 

-.65 

.43 

.280 

-1.67 

.36 

3 

6.61  E-02 

.38 

.983 

-.84 

.97 

2 

1 

.65 

.43 

.280 

-.36 

1.67 

3 

.72 

.42 

.205 

-.28 

1.72 

3 

1 

-6.61  E-02 

.38 

.983 

-.97 

.84 

2 

-.72 

.42 

.205 

-1.72 

.28 

Based  on  observed  means. 


Homogeneous  Subsets 


PROFILES 

Tukey  HSD®'*’-® 


PROFSiON 

N 

Subset 

1 

3 

41 

4.17 

1 

38 

4.24 

27 

4.89 

.190 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square{Error)  =  2.852. 

a.  Uses  Harmonic  Mean  Sample  Size  =  34.191. 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


LEVLCMD 


Dependent  Variable:  PROFILES 
Tukey  HSD 


Multiple  Comparisons 


(1)  LEVLCMD 

Mean 

Difference 

(hJ) 

Std.  Error 

m 

95%  Confidence  interval 

Lower  Bound 

1 

2 

.45 

-1.08 

1.06 

3 

.86 

-1.41 

2.71 

2 

1 

1.18E-02 

.45 

-1.06 

1.08 

3 

.66 

.94 

-1.58 

2.90 

3 

1 

imiiiiimni 

2 

Based  on  observed  means. 


Homogeneous  Subsets 


PROFILES 

Tukey  HSD®’'’'‘= 


LEVLCMD 

N 

Subset 

1 

3 

4 

1 

85 

17 

4.41 

.675 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  2.852. 

a.  Uses  Harmonic  Mean  Sample  Size  =  9.358. 

b.  The  group  sizes  are  unequai.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  i  error  leveis  are  not  guaranteed. 

c.  Alpha  =  .05. 


Relative  Weighted  Product 
per  1000  Enrollees 


Appendix  T 
Tukeys  HSD 


Univariate  Anaiysis  of  Variance 


Between-Subjects  Factors 


N 

SERVICE  1 

25 

2 

24 

3 

33 

LEVLCMD  1 

63 

2 

15 

3 

4 

PROFSION  1 

29 

2 

15 

3 

38 

Tests  of  Between-Subjects  Effects 


Dependent  Variable:  RWPENROL 


Source 

df 

F 

Corrected  Model 

50.568® 

16 

3.161 

1.341 

.201 

Intercept 

878.855 

1 

878.855 

372.911 

.000 

SERVICE 

1.267 

2 

.633 

.269 

.765 

LEVLCMD 

9.105 

2 

4.553 

1.932 

.153 

PROFSION 

26.163 

2 

13.081 

5.551 

.006 

SERVICE  *  LEVLCMD 

4.037 

4 

1.009 

.428 

.788 

SERVICE  *  PROFSION 

.926 

4 

.232 

.098 

.983 

LEVLCMD  * 

PROFSION 

3.632 

1 

3.632 

1.541 

.219 

SERVICE  *  LEVLCMD  * 
PROFSION 

1.741 

1 

1.741 

.739 

.393 

Error 

153.188 

65 

2.357 

Total 

2002.000 

82 

Corrected  Total 

203.756 

81 

a.  R  Squared  =  .248  (Adjusted  R  Squared  =  .063) 


Post  Hoc  Tests 


SERVICE 


Dependent  Variable:  RWPENROL 
Tukey  HSD 


Multiple  Comparisons 


(J)  SERVICE 

Mean 

Difference 

(l-J) 

Std.  Error 

sig. 

95%  Confidence  Interval 

Lower  Bound 

Upper  Bound 

1 

2 

-.44 

.44 

.573 

-1.50 

.61 

3 

.22 

.41 

.857 

-.76 

1.19 

2 

1 

.44 

.44 

.573 

-.61 

1.50 

3 

.66 

.41 

.253 

CO 

CO 

r 

1.65 

3 

1 

-.22 

.41 

.857 

-1.19 

.76 

2 

-.66 

.41 

.253 

-1.65 

.33 

Based  on  observed  means. 


Homogeneous  Subsets 


RWPENROL 


Tukey 


SERVICE 

N 

Subset 

1 

3 

33 

4.42 

1 

25 

4.64 

24 

5.08 

.265 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square{Error)  =  2.357. 

a.  Uses  Harmonic  Mean  Sample  Size  =  26.793. 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


PROFSION 


Dependent  Variable:  RWPENROL 
Tukey  HSD 


Multiple  Comparisons 


(1)  PROFSION 

(J)  PROFSION 

Mean 

Difference 

(l-J) 

Std.  Error 

Sig. 

95%  Confidence  Interval 

Lower  Bound 

Upper  Bound 

1 

2 

-.91 

.49 

.159 

-2.08 

.26 

3 

.52 

.38 

.358 

-.39 

1.43 

2 

1 

.91 

.49 

.159 

-.26 

2.08 

3 

1.43* 

.47 

.009 

.31 

2.55 

3 

1 

-.52 

.38 

.358 

-1.43 

.39 

2 

-1.43* 

.47 

.009 

-2.55 

-.31 

Based  on  observed  means. 

*.  The  mean  difference  is  significant  at  the  .05  level. 


Homogeneous  Subsets 


RWPENROL 

Tukey 


PROFSION 

N 

Subset  1 

1 

2 

3 

38 

4.24 

1 

29 

4.76 

4.76 

15 

5.67 

.478 

.113 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  2.357. 


a.  Uses  Harmonic  Mean  Sample  Size  =  23.536. 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


LEVLCMD 


Dependent  Variable:  RWPENROL 
Tukey  HSD 


Multiple  Comparisons 


(I)  LEVLCMD 

(J)  LEVLCMD 

Mean 

Difference 

(l-J) 

Std.  Error 

Sig. 

95%  Confidence  Interval 

Lower  Bound 

Upper  Bound 

1 

2 

1.90E-02 

.44 

.999 

-1.04 

1.08 

3 

-1.38 

.79 

.197 

-3.28 

.52 

2 

1 

-1.90E-02 

.44 

.999 

-1.08 

1.04 

3 

-1.40 

.86 

.244 

-3.47 

.67 

3 

1 

1.38 

.79 

.197 

-.52 

3.28 

2 

1.40 

.86 

.244 

-.67 

3.47 

Based  on  observed  means. 


Homogeneous  Subsets 


RWPENROL 

Tukey  HSD®''’-® 


LEVLCMD 

N 

Subset 

1 

2 

15 

4.60 

1 

63 

4.62 

4 

6.00 

.137 

Means  for  groups  in  homogeneous  subsets  are  dispiayed. 

Based  on  Type  ili  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  2.357. 

a.  Uses  Harmonic  Mean  Sample  Size  =  9.021. 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


MCSC  Phones,  %  of  all  Lines  Busy 


Appendix  U 
Tukey's  HSD 


Univariate  Analysis  of  Variance 


Between-Subjects  Factors 


N 

SERVICE  1 

32 

2 

28 

3 

43 

LEVLCMD  1 

83 

2 

16 

3 

4 

PROFSION  1 

38 

2 

25 

3 

40 

Tests  of  Between-Subjects  Effects 


Dependent  Variable;  MCSPHONE 


Source 

df 

F 

mm 

Corrected  Model 

45.483® 

17 

2.675 

1.227 

.262 

Intercept 

632.991 

1 

632.991 

290.251 

.000 

SERVICE 

5.236 

2 

2.618 

1.201 

.306 

LEVLCMD 

4.349 

2 

2.174 

.997 

.373 

PROFSION 

2.872 

2 

1.436 

.658 

.520 

SERVICE  *  LEVLCMD 

21.738 

4 

5.435 

2.492 

.049 

SERVICE  •  PROFSION 

12.908 

4 

3.227 

1.480 

.216 

LEVLCMD  * 

PROFSION 

4.590 

2 

2.295 

1.052 

.354 

SERVICE  *  LEVLCMD  * 
PROFSION 

3.393E-04 

1 

3.393E-04 

.000 

.990 

Error 

185.371 

2.181 

Total 

2629.000 

Corrected  Total 

230.854 

102 

a.  R  Squared  =  .197  (Adjusted  R  Squared  =  .036) 


Post  Hoc  Tests 


SERVICE 


Dependent  Variable:  MCSPHONE 
Tukey  HSD 


Multiple  Comparisons 


(J)  SERVICE 

Mean 

Difference 

f-J) _ 

Std.  Error 

Sig. 

95%  Confidence  intervai 

Lower  Bound 

Upper  Bound 

1 

2 

.42 

.38 

.525 

-.50 

1.33 

3 

.37 

.34 

.528 

-.45 

1.20 

2 

1 

-.42 

.38 

.525 

-1.33 

.50 

3 

-4.24E-02 

.36 

.992 

-.90 

.81 

3 

1 

-.37 

.34 

.528 

-1.20 

.45 

2 

4.24E-02 

.36 

.992 

-.81 

.90 

Based  on  observed  means. 


Homogeneous  Subsets 


MCSPHONE 

Tukey 


SERViCE 

N 

Subset 

1 

2 

28 

4.68 

3 

43 

4.72 

1 

32 

5.09 

Sig. 

.488 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  2.181. 

a.  Uses  Harmonic  Mean  Sample  Size  =  33.252. 

b.  The  group  sizes  are  unequai.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  i  error  ievels  are  not  guaranteed. 

c.  Aipha  =  .05. 


PROFSION 


Dependent  Variable:  MCSPHONE 
Tukey  HSD 


Multiple  Comparisons 


(I)  PROFSION 

(J)  PROFSION 

Mean 

Difference 

(l-J) 

Std.  Error 

■ 

95%  Confidence  Interval 

Lower  Bound 

■iffifsisfsnRfn 

1 

2 

-.25 

.38 

.791 

-1.16 

.66 

3 

-.34 

.33 

.562 

-1.14 

.45 

2 

1 

.25 

.38 

.791 

-.66 

1.16 

3 

-9.50E-02 

.38 

.966 

-.99 

.80 

3 

1 

.34 

-.45 

1.14 

2 

9.50E-02 

-.80 

.99 

Based  on  observed  means. 


Homogeneous  Subsets 


MCSPHONE 

Tukey  HSD®’*’  '' 


PROFSION 

N 

Subset 

1 

1 

40 

4.97 

.615 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  2.181. 

a.  Uses  Harmonic  Mean  Sample  Size  =  32.853. 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


LEVLCMD 


Dependent  Variable:  MCSPHONE 
Tukey  HSD 


Multiple  Comparisons 


(i)  LEVLCMD 

(J)  LEVLCMD 

Mean 

Difference 

(l-J) 

Std.  Error 

Sig. 

95%  Confidence  intervai 

Lower  Bound 

Upper  Bound 

1 

2 

-.37 

.40 

.633 

-1.33 

.59 

3 

1.32 

.76 

.195 

-.48 

3.12 

2 

1 

.37 

.40 

.633 

-.59 

1.33 

3 

1.69 

.83 

.108 

-.28 

3.66 

3 

1 

-1.32 

.76 

.195 

-3.12 

.48 

2 

-1.69 

.83 

.108 

-3.66 

.28 

Based  on  observed  means. 


Homogeneous  Subsets 


MCSPHONE 


Tukey 


LEVLCMD 

N 

Subset  1 

1 

2 

3 

4 

3.50 

1 

83 

4.82 

4.82 

16 

5.19 

.139 

.854 

Means  for  groups  in  homogeneous  subsets  are  dispiayed. 

Based  on  Type  iii  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  2.181. 


a.  Uses  Harmonic  Mean  Sampie  Size  =  9.244. 

b.  The  group  sizes  are  unequai.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  i  error  ieveis  are  not  guaranteed. 

c.  Aipha  =  .05. 


Malpractice  Claims  Filed  per  100  Physicians 


Appendix  V 
Tukey’s  HSD 


Univariate  Anaiysis  of  Variance 


Between-Subjects  Factors 


N 

SERVICE  1 

34 

2 

31 

3 

46 

LEVLCMD  1 

89 

2 

17 

3 

5 

PROFSION  1 

40 

2 

28 

3 

43 

Tests  of  Between-Subjects  Effects 
Dependent  Variable:  MALPRCFL 


Source 

df 

F 

mm 

Corrected  Model 

78.502® 

17 

4.618 

1.566 

.090 

Intercept 

450.642 

1 

450.642 

152.803 

.000 

SERVICE 

.692 

2 

.346 

.117 

.889 

LEVLCMD 

6.388 

2 

3.194 

1.083 

.343 

PROFSION 

10.798 

2 

5.399 

1.831 

.166 

SERVICE  *  LEVLCMD 

.163 

4 

4.078E-02 

.014 

1.000 

SERVICE  *  PROFSION 

34.716 

4 

8.679 

2.943 

.024 

LEVLCMD  * 

PROFSION 

5.326 

2 

2.663 

.903 

.409 

SERVICE  *  LEVLCMD  * 
PROFSION 

.162 

1 

.162 

.055 

.815 

Error 

274.273 

93 

2.949 

Total 

2169.000 

111 

Corrected  Total 

352.775 

110 

a.  R  Squared  =  .223  (Adjusted  R  Squared  =  .080) 


Post  Hoc  Tests 


SERViCE 


Dependent  Variable:  MALPRCFL 
Tukey  HSD 


Multiple  Comparisons 


H 

(J)  SERVICE 

Mean 

Difference 

(l-J) 

Std.  Error 

Sig. 

95%  Confidence  Interval 

Lower  Bound 

Upper  Bound 

1 

2 

-7.59E-02 

.43 

.983 

-1.09 

.94 

3 

.23 

.39 

.830 

-.70 

1.15 

2 

1 

7.59E-02 

.43 

.983 

-.94 

1.09 

3 

.30 

.40 

.730 

-.65 

1.25 

3 

1 

-.23 

.39 

.830 

-1.15 

.70 

2 

-.30 

.40 

.730 

-1.25 

.65 

Based  on  observed  means. 


Homogeneous  Subsets 


MALPRCFL 


Tukey  HSD®'*’-® 


SERVICE 

N 

Subset 

1 

3 

46 

3.89 

1 

34 

4.12 

31 

4.19 

.737 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  2.949. 

a.  Uses  Harmonic  Mean  Sample  Size  =  35.967. 

b.  The  group  sizes  are  unequai.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


PROFSION 


Dependent  Variable:  MALPRCFL 
Tukey  HSD 


Multiple  Comparisons 


(1)  PROFSION 

(J)  PROFSION 

Mean 

Difference 

(l-J) 

Std.  Error 

■ 

95%  Confidence  Interval 

Lower  Bound 

1 

2 

-.89 

.42 

.093 

.12 

3 

.47 

.38 

.437 

1.36 

2 

1 

.89 

.42 

.093 

-.12 

1.90 

3 

1.36* 

.42 

.004 

.36 

2.35 

3 

1 

-.47 

.38 

.437 

-1.36 

.43 

2 

-1.36* 

.42 

-2.35 

-.36 

Based  on  observed  means. 

*.  The  mean  difference  is  significant  at  the  .05  level. 


Homogeneous  Subsets 


MALPRCFL 


Tukey 


PROFSION 

N 

Subset  1 

1 

2 

3 

1 

^■a 

4.00 

28 

4.89 

.489 

.077 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  2.949. 


a.  Uses  Harmonic  Mean  Sample  Size  =  35.727. 


b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


LEVLCMD 


Dependent  Variable:  MALPRCFL 
Tukey  HSD 


Multiple  Comparisons 


(I)  LEVLCMD 

(J)  LEVLCMD 

Mean 

Difference 

(1-J) 

Std.  Error 

Sig. 

95%  Confidence  Intervai 

Lower  Bound 

Upper  Bound 

1 

2 

.51 

.45 

.503 

-.57 

1.59 

3 

.76 

.79 

.604 

-1.12 

2.64 

2 

1 

-.51 

.45 

.503 

-1.59 

.57 

3 

.25 

.87 

.957 

-1.83 

2.33 

3 

1 

-.76 

.79 

.604 

-2.64 

1.12 

2 

-.25 

.87 

.957 

-2.33 

1.83 

Based  on  observed  means. 


Homogeneous  Subsets 


MALPRCFL 

Tukey  HSD®’*’-® 


LEVLCMD 

N 

Subset 

1 

3 

5 

3.40 

2 

17 

3.65 

1 

89 

4.16 

Sig. 

.554 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  2.949. 

a.  Uses  Harmonic  Mean  Sample  Size  =  1 1.109. 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  i  error  ievels  are  not  guaranteed. 

c.  Aipha  =  .05. 


Dental  Wellness 


Appendix  W 
TukeysHSD 


Univariate  Anaiysis  of  Variance 


Between-Subjects  Factors 


N 

SERVICE  1 

32 

2 

30 

3 

47 

LEVLCMD  1 

87 

2 

17 

3 

5 

PROFSION  1 

40 

2 

28 

3 

41 

Tests  of  Between-Subjects  Effects 
Dependent  Variable:  DENWELL 


Source 

■iMIliillliM 

df 

Li. 

■■ 

Corrected  Model 

81.067® 

17 

4.769 

2.279 

.007 

Intercept 

673.340 

1 

673.340 

321.846 

.000 

SERVICE 

.626 

2 

.313 

.150 

.861 

LEVLCMD 

5.200 

2 

2.600 

1.243 

.293 

PROFSION 

2 

1.922 

.919 

.403 

SERVICE  *  LEVLCMD 

4 

4.605 

2.201 

.075 

SERVICE  *  PROFSION 

4 

6.087 

2.909 

.026 

LEVLCMD  * 

PROFSION 

.170 

2 

8.503E-02 

.041 

.960 

SERVICE  *  LEVLCMD  * 
PROFSION 

13.672 

1 

13.672 

6.535 

.012 

Error 

190.383 

2.092 

Total 

3128.000 

Corrected  Total 

271.450 

108 

a.  R  Squared  =  .299  (Adjusted  R  Squared  =  .168) 


Post  Hoc  Tests 


SERViCE 


Dependent  Variable;  DENWELL 
Tukey  HSD 


Multiple  Comparisons 


(J)  SERVICE 

Mean 

Difference 

(l-J) 

Std.  Error 

Sig. 

95%  Confidence  interval 

Lower  Bound 

Upper  Bound 

1 

2 

.10 

.37 

.958 

-.77 

.98 

3 

-.41 

.33 

.427 

-1.20 

.38 

2 

1 

-.10 

.37 

.958 

-.98 

.77 

3 

-.52 

.34 

.283 

-1.32 

.29 

3 

1 

.41 

.33 

.427 

-.38 

1.20 

2 

.52 

.34 

.283 

-.29 

1.32 

Based  on  observed  means. 


Homogeneous  Subsets 


DENWELL 

Tukey  HSD®’’’'® 


SERVICE 

N 

Subset 

1 

2 

30 

4.87 

1 

32 

4.97 

47 

5.38 

.300 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  2.092. 

a.  Uses  Harmonic  Mean  Sample  Size  =  34.941. 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  i  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


PROFSION 


Dependent  Variable:  DENWELL 
Tukey  HSD 


Multiple  Comparisons 


(I)  PROFSION 

(J)  PROFSION 

Mean 

Difference 

(l-J) 

Std.  Error 

Sig. 

95%  Confidence  Interval 

Lower  Bound 

Upper  Bound 

1 

2 

-.38 

.36 

.540 

-1.23 

.47 

3 

.74 

.32 

.061 

-2.69E-02 

1.50 

2 

1 

.38 

.36 

.540 

-.47 

1.23 

3 

1.12* 

.35 

.006 

.27 

1.96 

3 

1 

-.74 

.32 

.061 

-1.50 

2.69E-02 

2 

-1.12* 

.35 

.006 

-1.96 

-.27 

Based  on  observed  means. 

*.  The  mean  difference  is  significant  at  the  .05  level. 


Homogeneous  Subsets 


DENWELL 


Tukey 


PROFSION 

N 

Subset  1 

1 

2 

3 

41 

4.56 

1 

40 

5.30 

5.30 

28 

5.68 

.087 

.517 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square{Error)  =  2.092. 

a.  Uses  Harmonic  Mean  Sample  Size  =  35.251 . 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


LEVLCMD 


Dependent  Variable:  DENWELL 
Tukey  HSD 


Multiple  Comparisons 


(I)  LEVLCMD 

(J)  LEVLCMD 

Mean 

Difference 

(l-J) 

Std.  Error 

Sig. 

95%  Confidence  Interval 

Lower  Bound 

Upper  Bound 

1 

2 

1.11* 

.38 

.013 

.20 

2.02 

3 

1.14 

.67 

.203 

-.44 

2.73 

2 

1 

-1.11* 

.38 

.013 

-2.02 

-.20 

3 

3.53E-02 

.74 

.999 

-1.72 

1.79 

3 

1 

-1.14 

.67 

.203 

-2.73 

.44 

2 

-3.53E-02 

.74 

.999 

-1.79 

1.72 

Based  on  observed  means. 

*.  The  mean  difference  is  significant  at  the  .05  ievel. 


Homogeneous  Subsets 


DENWELL 

Tukey  HSD®-*’’® 


LEVLCMD 

N 

Subset 

1 

3 

5 

4.20 

2 

17 

4.24 

1 

87 

5.34 

Siq. 

.155 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  2.092. 

a.  Uses  Harmonic  Mean  Sample  Size  =  1 1 .098. 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


Dental  Readiness 


Appendix  X 
Tukeys  HSD 


Univariate  Analysis  of  Variance 


Between-Subjects  Factors 


N 

SERVICE  1 

32 

2 

30 

3 

47 

LEVLCMD  1 

87 

2 

17 

3 

5 

PROFSION  1 

40 

2 

28 

3 

41 

Tests  of  Between-Subjects  Effects 
Dependent  Variable:  DENREAD 


Source 

df 

F 

mm 

Corrected  Model 

75.062® 

17 

4.415 

2.573 

.002 

Intercept 

817.370 

1 

817.370 

476.234 

.000 

SERVICE 

1.531 

2 

.765 

.446 

.642 

LEVLCMD 

9.391 

2 

4.696 

2.736 

.070 

PROFSION 

.497 

2 

.249 

.145 

.865 

SERVICE  *  LEVLCMD 

16.550 

4 

4.138 

2.411 

.055 

SERVICE  •  PROFSION 

18.763 

4 

4.691 

2.733 

.034 

LEVLCMD  * 

PROFSION 

.222 

2 

.111 

.065 

.937 

SERVICE  *  LEVLCMD  * 
PROFSION 

8.872 

1 

8.872 

5.169 

.025 

Error 

156.185 

91 

1.716 

Total 

3534.000 

109 

Corrected  Total 

231.248 

108 

a.  R  Squared  =  .325  (Adjusted  R  Squared  =  .198) 


Post  Hoc  Tests 


SERVICE 


Dependent  Variable;  DENREAD 
Tukey  HSD 


Multiple  Comparisons 


■ 

Mean 

Difference 

_ M _ 

Std.  Error 

■ 

95%  Confidence  intervai 

Lower  Bound 

1 

2 

.33 

.753 

-1.03 

.55 

3 

.30 

.025 

-1.52 

-8.45E-02 

2 

1 

.24 

.33 

3 

-.56 

.31 

3 

1 

.80* 

.30 

.025 

8.45E-02 

1.52 

2 

.56 

.31 

.166 

-.17 

1.29 

Based  on  observed  means. 


*.  The  mean  difference  is  significant  at  the  .05  ievel. 


Homogeneous  Subsets 


DENREAD 

Tukey 


SERVICE 

N 

Subset  1 

1 

2 

1 

32 

5.09 

2 

30 

5.33 

5.33 

3 

47 

5.89 

Sig. 

.726 

.179 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  1.716. 

a.  Uses  Harmonic  Mean  Sampie  Size  =  34.941. 

b.  The  group  sizes  are  unequai.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  ievels  are  not  guaranteed. 

c.  Aipha  =  .05. 


PROFSION 


Dependent  Variable:  DENREAD 
Tukey  HSD 


Multiple  Comparisons 


(1)  PROFSiON 

(J)  PROFSiON 

Mean 

Difference 

(l-J) 

Std.  Error 

Sig. 

95%  Confidence  Interval 

Lower  Bound 

Upper  Bound 

1 

2 

8.57E-02 

.32 

.962 

-.68 

.85 

3 

.73* 

.29 

.038 

3.31  E-02 

1.42 

2 

1 

-8.57E-02 

.32 

.962 

-.85 

.68 

3 

.64 

.32 

.119 

-.12 

1.41 

3 

1 

-.73* 

.29 

.038 

-1.42 

-3.31  E-02 

2 

-.64 

.32 

.119 

-1.41 

.12 

Based  on  observed  means. 

*.  The  mean  difference  is  significant  at  the  .05  levei. 


Homogeneous  Subsets 


DENREAD 

Tukey  HSD®  *’-® 


PROFSION 

N 

Subset 

1 

3 

41 

5.07 

2 

28 

5.71 

1 

40 

5.80 

-Sia _ 

.057 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =  1.716. 

a.  Uses  Harmonic  Mean  Sample  Size  =  35.251. 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  ievels  are  not  guaranteed. 

c.  Aipha  =  .05. 


LEVLCMD 


Multiple  Comparisons 


Dependent  Variable:  DENREAD 
Tukey  HSD 


(1)  LEVLCMD 

fj)  LEVLCMD 

Mean 

Difference 

(l-J) 

Std.  Error 

Sig. 

95%  Confidence  Interval 

Lower  Bound 

Upper  Bound 

1 

2 

1.35* 

.35 

.001 

.52 

2.18 

3 

-.30 

.60 

.873 

-1.73 

1.14 

2 

1 

-1.35* 

.35 

.001 

-2.18 

-.52 

3 

-1.65* 

.67 

.040 

-3.24 

-5.90E-02 

3 

1 

.30 

.60 

.873 

-1.14 

1.73 

2 

1.65* 

.67 

.040 

5.90E-02 

3.24 

Based  on  observed  means. 

*.  The  mean  difference  is  significant  at  the  .05  level. 


Homogeneous  Subsets 


DENREAD 


Tukey  HSD®’*’-® 


LEVLCMD 

N 

Subset  1 

1 

2 

2 

17 

4.35 

1 

87 

5.70 

5 

6.00 

1.000 

.853 

Means  for  groups  in  homogeneous  subsets  are  displayed. 

Based  on  Type  III  Sum  of  Squares 

The  error  term  is  Mean  Square(Error)  =1.716. 

a.  Uses  Harmonic  Mean  Sample  Size  =  11.098. 

b.  The  group  sizes  are  unequal.  The  harmonic  mean  of  the  group  sizes  is  used.  Type  I  error  levels  are  not  guaranteed. 

c.  Alpha  =  .05. 


Satisfaction  with  Quality  of  Healthcare  Appendix  Y 1 

Frequencies 

Statistics 


SATQUALT 


Satisfaction  w/Quality  of  Healthcare 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


%  Beneficiaries  Satisfied  with  Customer  Service 


Appendix  Y2 


Frequencies 


Statistics 


BENSATCU 


N  Valid 

110 

Missing 

2 

Mean 

6.15 

Std.  Deviation 

1.07 

Percentiles  25 

6.00 

50 

6.00 

75 

7.00 

BENSATCU 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  2 

1 

.9 

.9 

.9 

3 

4 

3.6 

3.6 

4.5 

4 

3 

2.7 

2.7 

7.3 

5 

12 

10.7 

10.9 

18.2 

6 

40 

36.4 

54.5 

7 

50 

45.5 

100.0 

Total 

110 

98.2 

100.0 

Missing  System 

2 

1.8 

Totai 

112 

100.0 

%  Beneficiaries  Satis  w/Customer  Serv 


2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Patient  Satisfaction  with  Access 


Appendix  Y3 


Frequencies 


Statistics 


PTSATACC 


N  Valid 

112 

Missing 

0 

Mean 

6.13 

Std.  Deviation 

1.04 

Percentiles  25 

6.00 

50 

6.00 

75 

7.00 

PTSATACC 


Frequency 

Percent 

Vaiid  Percent 

Cumulative 

Percent 

Valid  1 

1 

.9 

.9 

.9 

3 

1 

.9 

.9 

1.8 

4 

6 

5.4 

5.4 

7.1 

5 

15 

13.4 

13.4 

20.5 

6 

40 

35.7 

35.7 

56.3 

7 

49 

43.8 

43.8 

100.0 

Totai 

112 

100.0 

100.0 

Patient  Satisfaction  w/Access 

60  n - 


50*^ 


Likert  Scale  Values 


Satisfaction  with  Access  to  Healthcare 


Appendix  Y4 


Frequencies 


Statistics 


SATACHLT 


N  Valid 

112 

Missing 

0 

Mean 

6.09 

Std.  Deviation 

1.11 

Percentiles  25 

6.00 

50 

6.00 

75 

7.00 

SATACHLT 


Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

1 

.9 

.9 

.9 

2 

2 

1.8 

1.8 

2.7 

3 

1 

.9 

.9 

3.6 

4 

2 

1.8 

1.8 

5.4 

5 

17 

15.2 

15.2 

20.5 

6 

42 

37.5 

37.5 

58.0 

7 

47 

42.0 

42.0 

100.0 

Total 

112 

100.0 

100.0 

Satisfaction  w/Access  to  Healthcare 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Overall  Satisfaction  with  Waiting  Times 


Appendix  Y5 


Frequencies 


Statistics 


SATCLNVI 


N  Valid 

111 

Missing 

1 

Mean 

6.07 

Std.  Deviation 

1.04 

Percentiles  25 

6.00 

50 

6.00 

75 

7.00 

SATCLNVI 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  2 

2 

1.8 

1.8 

1.8 

4 

8 

7.1 

7.2 

9.0 

5 

12 

10.7 

10.8 

19.8 

6 

45 

40.2 

40.5 

60.4 

7 

44 

39.3 

39.6 

100.0 

Total 

111 

99.1 

100.0 

Missing  System 

1 

.9 

Total 

112 

100.0 

Overall  Satisfaction  w/Clinic  Visit 


42 

c 

0 

■Q 

c 

o 

Q. 

(0 

0 

a: 


0 

E 

3 

z 


Std.  Dev  =  1 .04 
Mean  =  6.1 
N  =  111.00 


Likert  Scale  Values 


Patient  Satisfaction  with  Waiting  Times 


Appendix  Y6 


Frequencies 


Statistics 


PTSATWAI 


N  Valid 

112 

Missing 

0 

Mean 

5.94 

Std.  Deviation 

1.16 

Percentiles  25 

5.25 

50 

6.00 

75 

7.00 

PTSATWAI 


Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

1 

.9 

.9 

.9 

3 

4 

3.6 

3.6 

4.5 

4 

8 

7.1 

7.1 

11.6 

5 

15 

13.4 

13.4 

25.0 

6 

43 

38.4 

38.4 

63.4 

7 

41 

36.6 

36.6 

100.0 

Total 

112 

100.0 

100.0 

Patient  Satisfaction  w/Waiting  Times 

50 - 1 


Likert  Scale  Values 


%  Eligibles  Enrolled  to  MTF 


Appendix  Y7 


Frequencies 


Statistics 


ELIGENRL 


N  Valid 

111 

Missing 

1 

Mean 

5.91 

Std.  Deviation 

1.25 

Percentiles  25 

5.00 

50 

6.00 

75 

7.00 

ELIGENRL 


Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

2 

1.8 

1.8 

1.8 

2 

1 

.9 

.9 

2.7 

3 

2 

1.8 

'  1.8 

4.5 

4 

6 

5.4 

5.4 

9.9 

5 

22 

19.6 

19.8 

29.7 

6 

34 

30.4 

30.6 

60.4 

7 

44 

39.3 

39.6 

100.0 

Total 

111 

99.1 

100.0 

Missing  System 

1 

.9 

Total 

112 

100.0 

%  Eligibles  Enrolled  to  MTF 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Cost  per  Enrollee 


Appendix  Y8 


Frequencies 


Statistics 


COSTENRL 


N  Valid 

110 

Missing 

2 

Mean 

5.89 

Std.  Deviation 

1.15 

Percentiies  25 

5.00 

50 

6.00 

75 

7.00 

COSTENRL 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

1 

.9 

.9 

.9 

3 

2 

1.8 

1.8 

2.7 

4 

11 

9.8 

10.0 

12.7 

5 

19 

17.0 

17.3 

30.0 

6 

33.0 

33.6 

63.6 

7 

^B9 

35.7 

36.4 

100.0 

Total 

98.2 

100.0 

Missing  System 

1.8 

Total 

100.0 

Cost  per  Enrollee 

50  T - 


Likert  Scale  Values 


Ease  of  Making  an  Appointment  by  Phone 


Appendix  Y9 


Frequencies 


Statistics 


APPTEASE 


N  Valid 

111 

Missing 

1 

Mean 

5.89 

Std.  Deviation 

1.23 

Percentiies  25 

5.00 

50 

6.00 

75 

7.00 

APPTEASE 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

1 

.9 

.9 

.9 

2 

2 

1.8 

1.8 

2.7 

3 

2 

1.8 

1.8 

4.5 

4 

6 

5.4 

5.4 

9.9 

5 

25 

22.3 

22.5 

32.4 

6 

31 

27.7 

27.9 

60.4 

7 

44 

39.3 

39.6 

100.0 

Total 

111 

99.1 

100.0 

Missing  System 

1 

.9 

Total 

112 

100.0 

Ease  of  Making  Appt  by  Phone 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Emergency  Department  Visits  per  1000  Enrollees 


Appendix  YIO 


Frequencies 


Statistics 

ERVISITS 


N  Valid 

109 

Missing 

3 

Mean 

5.87 

Std.  Deviation 

1.33 

Percentiles  25 

5.00 

50 

6.00 

75 

7.00 

ERVISITS 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

3 

2.7 

2.8 

3 

5 

4.5 

4.6 

4 

3 

2.7 

2.8 

10.1 

5 

19 

17.0 

17.4 

6 

38 

33.9 

34.9 

7 

41 

36.6 

37.6 

100.0 

Total 

109 

97.3 

100.0 

Missing  System 

3 

2.7 

Total 

112 

100.0 

Emergency  Dept  Visits/1000  Enrollees 

50  T - 1 


Likert  Scale  Values 


Preventive  Care  Measures 


Appendix  Yll 


Frequencies 


Statistics 


PREVCARE 


N  Valid 

111 

Missing 

1 

Mean 

5.81 

Std.  Deviation 

1.18 

Percentiles  25 

5.00 

50 

6.00 

75 

7.00 

PREVCARE 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  2 

1 

.9 

.9 

.9 

3 

6 

5.4 

5.4 

6.3 

4 

8 

7.1 

7.2 

13.5 

5 

19 

17.0 

17.1 

30.6 

6 

41 

36.6 

67.6 

7 

36 

32.1 

100.0 

Total 

111 

99.1 

100.0 

Missing  System 

1 

.9 

Totai 

112 

100.0 

Preventive  Care  Measures 


2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Pharmacy  Cost  per  Member  per  Month 


Appendbp^^rl^ y  /  2, 


Frequencies 


Statistics 


PHRMCOST 


N  Valid 

112 

Missing 

0 

Mean 

5.78 

Std.  Deviation 

1.35 

Percentiles  25 

5.00 

50 

6.00 

75 

7.00 

PHRMCOST 


Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

3 

2.7 

2.7 

2.7 

2 

2 

1.8 

1.8 

4.5 

3 

2 

1.8 

1.8 

6.3 

4 

6 

5.4 

5.4 

11.6 

5 

21 

18.8 

18.8 

30.4 

*  6 

41 

36.6 

36.6 

67.0 

7 

37 

33.0 

33.0 

100.0 

Total 

112 

100.0 

100.0 

Pharmacy  Cost  PMPM 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Satisfaction  with  Access  to  Specialist 


Appendix  Y1 3 


Frequencies 


Statistics 


SATACSPE 


N  Valid 

112 

Missing 

0 

Mean 

5.70 

Std.  Deviation 

1.19 

Percentiles  25 

5.00 

50 

6.00 

75 

7.00 

SATACSPE 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

1 

.9 

.9 

.9 

2 

2 

1.8 

1.8 

2.7 

3 

2 

1.8 

1.8 

4.5 

4 

9 

8.0 

8.0 

12.5 

5 

27 

24.1 

24.1 

36.6 

6 

41 

36.6 

36.6 

73.2 

7 

30 

26.8 

26.8 

100.0 

Total 

112 

100.0 

100.0 

Satisfaction  w/Access  to  Specialist 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Direct  Care  Market  Share  -  Outpatient  Workload 


Appendix  Y1 4 


Frequencies 


Statistics 


DCMKTOUT 


N  Valid 

107 

Missing 

5 

Mean 

5.69 

Std.  Deviation 

1.40 

Percentiles  25 

5.00 

50 

6.00 

75 

7.00 

DCMKTOUT 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

3 

2.7 

2.8 

2.8 

2 

3 

2.7 

2.8 

5.6 

3 

2 

1.8 

1.9 

7.5 

4 

6 

5.4 

5.6 

13.1 

5 

21 

18.8 

19.6 

32.7 

6 

39 

34.8 

36.4 

69.2 

7 

33 

29.5 

30.8 

100.0 

Total 

107 

95.5 

100.0 

Missing  System 

5 

4.5 

Total 

112 

Direct  Care  Market  Share-Outpt 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Wait  Time  for  Appointment  (TRICARE  Standards) 


Appendix  Y15 


Frequencies 


Statistics 


WAITFAPP 


N  Valid 

112 

Missing 

0 

Mean 

5.62 

Std.  Deviation 

1.34 

Percentiles  25 

5.00 

50 

6.00 

75 

7.00 

WAITFAPP 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

2 

1.8 

1.8 

1.8 

2 

1 

.9 

.9 

2.7 

3 

3.6 

3.6 

6.3 

4 

13.4 

13.4 

19.6 

5 

18.8 

18.8 

38.4 

6 

31.3 

31.3 

69.6 

7 

34 

30.4 

30.4 

100.0 

Total 

112 

100.0 

100.0 

Wait  Time  FOR  Appointment 

40 - 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


%  Enrollees  Seen  by  their  PCM 


Appendix  Y16 


Frequencies 


Statistics 


ENRPCM 


N  Valid 

112 

Missing 

0 

Mean 

5.58 

Std.  Deviation 

1.36 

Percentiles  25 

5.00 

50 

6.00 

75 

7.00 

ENRPCM 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

3 

2.7 

2.7 

2.7 

2 

2 

1.8 

1.8 

4.5 

3 

2 

1.8 

1.8 

6.3 

4 

12 

10.7 

10.7 

17.0 

5 

23 

20.5 

6 

41 

36.6 

7 

29 

25.9 

25.9 

Total 

112 

100.0 

100.0 

HHIHH 

Likert  Scale  Values 


Wait  Time  at  Appointment  (30  minute  standard) 


Appendix  Y17 


Frequencies 


Statistics 

WAITAAPP 


N  Valid 

112 

Missing 

0 

Mean 

5.54 

Std.  Deviation 

1.15 

Percentiles  25 

5.00 

50 

6.00 

75 

6.00 

WAITAAPP 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  2 

1 

.9 

.9 

.9 

3 

3.6 

3.6 

4.5 

4 

12.5 

12.5 

17.0 

5 

30.4 

30.4 

47.3 

6 

28.6 

28.6 

75.9 

7 

24.1 

24.1 

100.0 

Total 

112 

100.0 

100.0 

Wait  Time  AT  Appointment 


40 


Lik« 


Visits  per  Provider  per  Day 


Appendix  Y1 8 


Frequencies 


Statistics 


ViSPPROV 


N  Valid 

112 

Missing 

0 

Mean 

5.53 

Std.  Deviation 

1.24 

Percentiles  25 

5.00 

50 

6.00 

75 

6.00 

VISPPROV 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  2 

3 

2.7 

2.7 

2.7 

3 

5 

4.5 

4.5 

7.1 

4 

12 

10.7 

10.7 

5 

25.0 

25.0 

6 

33.9 

33.9 

76.8 

7 

26 

23.2 

23.2 

100.0 

Total 

112 

100.0 

100.0 

Visits  per  Provider  per  Day 

40 1 - 


2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Cost  per  Visit  (Weighted) 


Appendix  Y19 


Frequencies 


Statistics 


COSTVIST 


N  Valid 

105 

Missing 

7 

Mean 

5.52 

Std.  Deviation 

1.39 

Percentiles  25 

5.00 

50 

6.00 

75 

7.00 

COSTViST 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

3 

2.7 

2.9 

2.9 

3 

7 

6.3 

6.7 

9.5 

4 

8 

7.1 

7.6 

17.1 

5 

26 

23.2 

24.8 

41.9 

6 

33 

29.5 

31.4 

73.3 

7 

28 

25.0 

26.7 

100.0 

Total 

105 

93.8 

100.0 

Missing  System 

7 

6.3 

Total 

112 

100.0 

Cost  per  Visit  Weighted 

40  T - - - 


Likert  Scale  Values 


Std.  Dev  =  1.39 
Mean  =  5.5 
N  =  105.00 


Dental  Readiness  (%  Class  1  and  2) 


Appendix  Y20 


Frequencies 


Statistics 

DENREAD 


N  Valid 

109 

Missing 

3 

Mean 

5.50 

Std.  Deviation 

1.46 

Percentiles  25 

5.00 

50 

6.00 

75 

7.00 

DENREAD 


%  Providers  Medical  Readiness  Trained  and  Certified 


Appendix  Y21 


Frequencies 


Statistics 


PROVCERT 


N  Valid 

112 

Missing 

0 

Mean 

5.41 

Std.  Deviation 

1.59 

Percentiles  25 

5.00 

50 

6.00 

75 

7.00 

PROVCERT 


Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

3 

2.7 

2.7 

2.7 

2 

4 

3.6 

3.6 

6.3 

3 

8 

7.1 

7.1 

13.4 

4 

12 

10.7 

10.7 

24.1 

5 

23 

20.5 

20.5 

44.6 

6 

26 

23.2 

23.2 

67.9 

7 

36 

32.1 

32.1 

100.0 

Total 

112 

100.0 

100.0 

%  Providers  Med  Readiness 
Trained  and  Certified 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Cost  per  Bedday 


Appendix  Y22 


Frequencies 


Statistics 

COSTBDDY 


N  Valid 

110 

Missing 

2 

Mean 

5.39 

Std.  Deviation 

1.38 

Percentiles  25 

5.00 

50 

6.00 

75 

6.00 

COSTBDDY 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

2 

1.8 

1.8 

2 

3 

2.7 

4.5 

3 

6 

5.5 

10.0 

4 

13 

11.6 

11.8 

21.8 

5 

23 

20.5 

20.9 

42.7 

6 

41 

36.6 

37.3 

80.0 

7 

22 

19.6 

20.0 

100.0 

Total 

110 

98.2 

100.0 

Missing  System 

2 

1.8 

Total 

112 

100.0 

Likert  Scale  Values 


Preventable  Admission  Rates  for  Prime  Enrollees 


Appendix  Y23 


Frequencies 


Statistics 


PREVADM 


N  Valid 

109 

Missing 

3 

Mean 

5.35 

Std.  Deviation 

1.45 

Percentiles  25 

5.00 

50 

6.00 

75 

6.00 

PREVADM 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

2 

1.8 

1.8 

2 

4 

3.6 

5.5 

3 

6.3 

11.9 

4 

11.6 

11.9 

23.9 

5 

18.8 

19.3 

43.1 

6 

34.8 

78.9 

7 

23 

20.5 

100.0 

Total 

109 

97.3 

100.0 

Missing  System 

3 

2.7 

Total 

112 

100.0 

Preventable  Admission  Rates  for 
Prime  Enrollees 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Cost  per  RWP 


Appendix  Y24 


Frequencies 


Statistics 


COSTRWP 


N  Valid 

98 

Missing 

14 

Mean 

5.34 

Std.  Deviation 

1.47 

Percentiles  25 

4.75 

50 

6.00 

75 

6.00 

COSTRWP 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

3 

2.7 

3.1 

3.1 

2 

2 

1.8 

2.0 

5.1 

3 

6 

5.4 

6.1 

11.2 

4 

13 

11.6 

13.3 

24.5 

5 

19 

17.0 

19.4 

43.9 

6 

34 

30.4 

34.7 

78.6 

7 

21 

18.8 

21.4 

100.0 

Total 

98 

87.5 

100.0 

Missing  System 

14 

12.5 

Total 

112 

100.0 

Cost  per  Relative  Weighted  Product 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Claims  Processed  within  30  Days 


Appendix  Y25 


Frequencies 


Statistics 


CLMSPROC 


N  Valid 

112 

Missing 

0 

Mean 

5.32 

Std.  Deviation 

1.43 

Percentiles  25 

4.00 

50 

6.00 

75 

6.00 

CLMSPROC 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

1 

.9 

.9 

.9 

2 

5 

4.5 

4.5 

5.4 

3 

6 

5.4 

5.4 

10.7 

4 

19 

17.0 

17.0 

27.7 

5 

20 

17.9 

17.9 

45.5 

6 

36 

32.1 

32.1 

77.7 

7 

25 

22.3 

22.3 

100.0 

Total 

112 

100.0 

100.0 

Claims  Processed  within  30  days 

40  T - , 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Visits  per  1000  weighted 


Appendix  Y26 


Frequencies 


Statistics 


VISP1000 


N  Valid 

103 

Missing 

9 

Mean 

5.15 

Std.  Deviation 

1.34 

Percentiles  25 

4.00 

50 

5.00 

75 

6.00 

VISP1000 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

2 

1.8 

1.9 

2 

4 

3.6 

5.8 

3 

3 

2.7 

8.7 

4 

18 

16.1 

17.5 

26.2 

5 

31 

27.7 

56.3 

6 

31 

27.7 

86.4 

7 

14 

12.5 

13.6 

100.0 

Totai 

103 

92.0 

100.0 

Missing  System 

9 

8.0 

Totai 

112 

100.0 

Visits  per  1 000  Weighted 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Dental  Wellness  (%  Class  1) 


Appendix  Y27 


Frequencies 


Statistics 


DENWELL 


N  Valid 

109 

Missing 

3 

Mean 

5.12 

Std.  Deviation 

1.59 

Percentiles  25 

4.00 

50 

5.00 

75 

6.00 

DENWELL 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

3 

2.7 

2.8 

2.8 

2 

8 

7.1 

7.3 

10.1 

3 

3 

2.7 

2.8 

12.8 

4 

20 

17.9 

18.3 

31.2 

5 

23 

20.5 

21.1 

52.3 

6 

29 

25.9 

26.6 

78.9 

7 

23 

20.5 

21.1 

100.0 

Total 

109 

97.3 

100.0 

Missing  System 

3 

2.7 

Total 

112 

100.0 

Dental  Wellness 


Likert  Scale  Values 


Direct  Care  Market  Share  -  Inpatient  Workload 


Appendix  Y28 


Frequencies 


Statistics 


DCMKTINP 


N  Valid 

105 

Missing 

7 

Mean 

5.08 

Std.  Deviation 

1.66 

Percentiles  25 

4.00 

50 

6.00 

75 

6.00 

DCMKTINP 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

6 

5.4 

5.7 

5.7 

2 

5 

4.5 

4.8 

10.5 

3 

6 

5.4 

5.7 

16.2 

4 

14 

12.5 

13.3 

29.5 

5 

19 

17.0 

18.1 

47.6 

6 

37 

33.0 

35.2 

82.9 

7 

18 

16.1 

17.1 

100.0 

Total 

105 

93.8 

100.0 

Missing  System 

7 

6.3 

Total 

112 

100.0 

Likert  Scale  Values 


MCSC  %  of  Calls  Answered  within  Two  Minutes 


Appendix  Y29 


Frequencies 


Statistics 

MCSCALLS 


N  Valid 

106 

Missing 

6 

Mean 

5.05 

Std.  Deviation 

1.41 

Percentiies  25 

4.00 

50 

5.00 

75 

6.00 

MCSCALLS 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

3 

2.7 

2.8 

2.8 

2 

3 

2.7 

2.8 

5.7 

3 

6 

5.4 

5.7 

11.3 

4 

21 

18.8 

19.8 

31.1 

5 

29 

25.9 

27.4 

58.5 

6 

29 

25.9 

27.4 

85.8 

7 

15 

13.4 

14.2 

100.0 

Total 

106 

94.6 

100.0 

Missing  System 

6 

5.4 

Total 

112 

MCSC  %  Calls  Answered  within  2  min 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Laboratory  Costs  per  Member  per  Month 


Appendix  Y30 


Frequencies 


Statistics 


LABCOST 


N  Valid 

107 

Missing 

5 

Mean 

5.02 

Std.  Deviation 

1.59 

Percentiles  25 

4.00 

50 

5.00 

75 

6.00 

LABCOST 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

4 

3.7 

3.7 

2 

6 

5.6 

9.3 

3 

8 

7.1 

7.5 

16.8 

4 

13 

11.6 

12.1 

29.0 

5 

31 

29.0 

57.9 

6 

25 

23.4 

81.3 

7 

20 

17.9 

18.7 

100.0 

Total 

107 

95.5 

100.0 

Missing  System 

5 

4.5 

Total 

112 

100.0 

Lab  Costs  PM  PM 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Radiology  Costs  per  Member  per  Month 


Appendix  Y31 


Frequencies 


Statistics 


RADCOST 


N  Valid 

111 

Missing 

1 

Mean 

4.96 

Std.  Deviation 

1.62 

Percentiles  25 

4.00 

50 

5.00 

75 

6.00 

RADCOST 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

4 

3.6 

3.6 

2 

7 

6.3 

9.9 

3 

9 

8.0 

8.1 

18.0 

4 

18 

16.1 

16.2 

34.2 

5 

24 

21.6 

55.9 

6 

29 

26.1 

82.0 

7 

20 

17.9 

18.0 

100.0 

Total 

111 

99.1 

100.0 

Missing  System 

1 

.9 

Total 

112 

100.0 

Radiology  Cost  PMPM 

40  T - 


30-^ 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


SADR  Completion  Rate 


Appendix  Y32 


Frequencies 


Statistics 


SADR 


N  Valid 

91 

Missing 

21 

Mean 

4.93 

Std.  Deviation 

1.53 

Percentiles  25 

4.00 

50 

5.00 

75 

6.00 

SADR 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

4 

3.6 

4.4 

4.4 

2 

4 

3.6 

4.4 

8.8 

3 

4 

3.6 

4.4 

13.2 

4 

19 

17.0 

20.9 

34.1 

5 

24 

26.4 

60.4 

6 

23 

25.3 

85.7 

7 

13 

11.6 

14.3 

100.0 

Total 

91 

81.3 

100.0 

Missing  System 

21 

18.8 

Total 

112 

100.0 

SADR  Completion  Rate 

30  T - 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


SIDR  Completion  Rate 


Appendix  Y33 


Frequencies 


Statistics 


SIDR 


N  Valid 

90 

Missing 

22 

Mean 

4.91 

Std.  Deviation 

1.56 

Percentiles  25 

4.00 

50 

5.00 

75 

6.00 

SIDR 


Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

5 

4.5 

5.6 

5.6 

2 

2 

1.8 

2.2 

7.8 

3 

5 

4.5 

5.6 

13.3 

4 

21 

18.8 

23.3 

36.7 

5 

22 

19.6 

24.4 

61.1 

6 

21 

18.8 

23.3 

84.4 

7 

14 

12.5 

15.6 

100.0 

Total 

90 

80.4 

100.0 

Missing  System 

22 

19.6 

Total 

112 

100.0 

SIDR  Completion  Rate 

30  T - 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Capital  Costs  per  Member  per  Month 


Appendix  Y34 


Frequencies 


Statistics 


CAPCOSTS 


N  Valid 

104 

Missing 

8 

Mean 

4.88 

Std.  Deviation 

1.74 

Percentiles  25 

4.00 

50 

5.00 

75 

6.00 

CAPCOSTS 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

7 

6.3 

6.7 

6.7 

2 

8 

7.1 

7.7 

14.4 

3 

3 

2.7 

2.9 

17.3 

4 

18 

16.1 

17.3 

34.6 

5 

21 

18.8 

20.2 

54.8 

6 

30 

26.8 

28.8 

83.7 

7 

17 

15.2 

16.3 

100.0 

Total 

104 

92.9 

100.0 

Missing  System 

8 

7.1 

Total 

112 

100.0 

Capital  Costs  PMPM 


Likert  Scale  Values 


Beddays  per  1000  Enrollees 


Appendix  Y35 


Frequencies 


Statistics 


BEDDAY 


N  Valid 

111 

Missing 

1 

Mean 

4.86 

Std.  Deviation 

1.68 

Percentiles  25 

4.00 

50 

5.00 

75 

6.00 

BEDDAY 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

7 

6.3 

6.3 

6.3 

2 

5 

4.5 

4.5 

10.8 

3 

10 

8.9 

9.0 

19.8 

4 

19 

17.0 

17.1 

36.9 

5 

20 

17.9 

18.0 

55.0 

6 

34 

30.4 

30.6 

85.6 

7 

16 

14.3 

14.4 

100.0 

Total 

111 

99.1 

100.0 

Missing  System 

1 

.9 

Total 

112 

100.0 

Bed  Days  per  1000  Enrollees 


Likert  Scale  Values 


MCSC  Phones,  %  of  all  lines  busy 


Appendix  Y36 


Frequencies 


Statistics 


MCSPHONE 


N  Valid 

103 

Missing 

9 

Mean 

4.83 

Std.  Deviation 

1.50 

Percentiles  25 

4.00 

50 

5.00 

75 

6.00 

MCSPHONE 


Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

3 

2.7 

2.9 

2.9 

2 

7 

6.3 

6.8 

9.7 

3 

7 

6.3 

6.8 

16.5 

4 

22 

19.6 

21.4 

37.9 

5 

24 

21.4 

23.3 

61.2 

6 

29 

25.9 

28.2 

89.3 

7 

11 

9.8 

10.7 

100.0 

Total 

103 

92.0 

100.0 

Missing  System 

9 

8.0 

Total 

112 

100.0 

MCSC  Phones  --  %  of  all  lines  busy 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Dispositions  per  1000  Enrollees 


Appendix  Y37 


Frequencies 


Statistics 


DISPENRO 


N  Valid 

107 

Missing 

5 

Mean 

4.78 

Std.  Deviation 

1.53 

Percentiies  25 

4.00 

50 

5.00 

75 

6.00 

DISPENRO 


Percent 

Valid  Percent 

Cumuiative 

Percent 

Vaiid  1 

4 

3.6 

3.7 

3.7 

2 

4 

3.6 

3.7 

7.5 

3 

13 

11.6 

12.1 

19.6 

4 

23 

20.5 

21.5 

41.1 

5 

22 

19.6 

20.6 

61.7 

6 

29 

25.9 

27.1 

88.8 

7 

12 

10.7 

11.2 

100.0 

Total 

107 

95.5 

100.0 

Missing  System 

5 

4.5 

Total 

112 

100.0 

Dispositions/1000  Enrollees 

40  T - 


30  ^ 


Likert  Scale  Values 


Nurse  Full  Time  Equivalent  per  Weighted  Bedday 


Appendix  Y38 


Frequencies 


Statistics 


NURSEFTE 


N  Valid 

108 

Missing 

4 

Mean 

4.74 

Std.  Deviation 

1.57 

Percentiles  25 

4.00 

50 

5.00 

75 

6.00 

NURSEFTE 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

6 

5.6 

5.6 

2 

5 

4.6 

10.2 

3 

7 

6.5 

16.7 

4 

24 

21.4 

22.2 

38.9 

5 

32 

28.6 

29.6 

68.5 

6 

19 

17.0 

17.6 

86.1 

7 

15 

13.4 

13.9 

100.0 

Total 

108 

96.4 

100.0 

Missing  System 

4 

3.6 

Total 

112 

100.0 

Nurse  FTE/Weighted  Bed  Day 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Beneficiary  Grievances  per  1000  Enrollees 


Appendix  Y39 


Frequencies 


Statistics 


BENGRIEV 


N  Valid 

110 

Missing 

2 

Mean 

4.70 

Std.  Deviation 

1.67 

Percentiles  25 

3.00 

50 

5.00 

75 

6.00 

BENGRIEV 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

3.6 

3.6 

2 

10.0 

13.6 

3 

11.6 

11.8 

25.5 

4 

12.5 

12.7 

38.2 

5 

24.5 

62.7 

6 

23.6 

86.4 

7 

15 

13.4 

13.6 

100.0 

Total 

110 

98.2 

100.0 

Missing  System 

2 

1.8 

Total 

112 

100.0 

Beneficiary  Grievances/1000  Enrollees 

30  T - , 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


RWPs  per  1000  Enrollees 


Appendix  Y40 


Frequencies 


Statistics 


RWPENROL 


N  Valid 

82 

Missing 

30 

Mean 

4.68 

Std.  Deviation 

1.59 

Percentiles  25 

4.00 

50 

5.00 

75 

6.00 

RWPENROL 


Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

5 

4.5 

6.1 

2 

6 

5.4 

7.3 

3 

3 

2.7 

3.7 

17.1 

4 

15 

13.4 

18.3 

5 

29 

25.9 

35.4 

6 

15 

13.4 

18.3 

89.0 

7 

9 

8.0 

11.0 

100.0 

Total 

82 

73.2 

100.0 

Missing  System 

30 

26.8 

Total 

112 

100.0 

Relative  Weighted  Products/1000  Enrollees 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Health  Status  (self  reported  on  SF  12  -  mean) 


Appendix  Y41 


Frequencies 


Statistics 


HLTHSTAT 


N  Valid 

104 

Missing 

8 

Mean 

4.64 

Std.  Deviation 

1.49 

Percentiles  25 

4.00 

50 

5.00 

75 

6.00 

HLTHSTAT 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

4 

3.6 

3.8 

3.8 

2 

4 

3.6 

3.8 

7.7 

3 

15 

13.4 

14.4 

22.1 

4 

21 

18.8 

42.3 

5 

28 

25.0 

69.2 

6 

22 

19.6 

21.2 

90.4 

7 

10 

8.9 

9.6 

100.0 

Total 

104 

92.9 

100.0 

Missing  System 

8 

7.1 

Total 

112 

100.0 

Health  Status 


Likert  Scale  Values 


MEPRS  Completion  Rate 


Appendix  Y42 


Frequencies 


Statistics 


MEPRS 


N  Valid 

111 

Missing 

1 

Mean 

4.41 

Std.  Deviation 

1.84 

Percentiles  25 

3.00 

50 

5.00 

75 

6.00 

MEPRS 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

9 

8.0 

8.1 

2 

15 

13.4 

21.6 

3 

6 

5.4 

27.0 

4 

25 

22.3 

22.5 

49.5 

5 

20 

17.9 

18.0 

67.6 

6 

20 

17.9 

18.0 

85.6 

7 

16 

14.3 

14.4 

100.0 

Total 

111 

99.1 

100.0 

Missing  System 

1 

.9 

Total 

112 

100.0 

MEPRS  Completion  Rate 

30  T - 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


%  of  AD  Members  with  Temporary  Medical  Profiles 


Appendix  Y43 


Frequencies 


Statistics 


PROFILES 


N  Valid 

106 

Missing 

6 

Mean 

4.38 

Std.  Deviation 

1.74 

Percentiles  25 

3.00 

50 

4.00 

75 

6.00 

PROFILES 


EsnuRi 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

8 

7.1 

7.5 

7.5 

2 

7 

6.3 

6.6 

14.2 

3 

19 

17.0 

17.9 

32.1 

4 

21 

18.8 

19.8 

51.9 

5 

18 

16.1 

17.0 

68.9 

6 

20 

17.9 

18.9 

87.7 

7 

13 

11.6 

12.3 

100.0 

Total 

106 

94.6 

100.0 

Missing  System 

6 

5.4 

Total 

112 

100.0 

%  Active  Duty  with  Temporary 
Medical  Profiles 

30  T - 1 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Beneficiary  Appeals  per  1000  Claims 


Appendix  Y44 


Frequencies 


Statistics 


BENAPPEA 


N  Valid 

107 

Missing 

5 

Mean 

4.21 

Std.  Deviation 

1.64 

Percentiles  25 

3.00 

50 

5.00 

75 

5.00 

BENAPPEA 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

5 

4.5 

4.7 

4.7 

2 

19 

17.0 

17.8 

22.4 

3 

11 

9.8 

10.3 

32.7 

4 

16 

14.3 

15.0 

47.7 

5 

32 

28.6 

29.9 

77.6 

6 

18 

16.1 

16.8 

94.4 

7 

6 

5.4 

5.6 

100.0 

Total 

107 

95.5 

100.0 

Missing  System 

5 

4.5 

Total 

112 

100.0 

Beneficiary  Appeals/1000  claims 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


JCAHO  Grid  Scores 


Appendix  Y45 


Frequencies 


Statistics 


JCAHOSCO 


N  Valid 

111 

Missing 

1 

Mean 

4.19 

Std.  Deviation 

1.76 

Percentiles  25 

3.00 

50 

4.00 

75 

5.00 

JCAHOSCO 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

10 

8.9 

9.0 

9.0 

2 

11 

9.8 

9.9 

18.9 

3 

15.2 

15.3 

34.2 

4 

19.6 

19.8 

54.1 

5 

21.4 

21.6 

75.7 

6 

15 

13.4 

13.5 

89.2 

7 

12 

10.7 

10.8 

100.0 

Totai 

111 

99.1 

100.0 

Missing  System 

1 

.9 

Total 

112 

100.0 

JCAHO  Grid  Scores 

30  T - 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Malpractice  Claims  Paid  per  100  Physicians 


Appendix  Y46 


Frequencies 


Statistics 


MALPRCPD 


N  Valid 

109 

Missing 

3 

Mean 

4.09 

Std.  Deviation 

1.78 

Percentiles  25 

3.00 

50 

4.00 

75 

5.00 

MALPRCPD 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

13 

11.6 

11.9 

11.9 

2 

11 

9.8 

10.1 

22.0 

3 

15 

13.4 

13.8 

35.8 

4 

17 

15.2 

15.6 

51.4 

5 

27 

24.1 

24.8 

76.1 

6 

19 

17.0 

17.4 

93.6 

7 

7 

6.3 

6.4 

100.0 

Total 

109 

97.3 

100.0 

Missing  System 

3 

2.7 

Total 

112 

100.0 

Malpractice  Claims  Paid/100  Physicians 

30t - , 


1.0  2.0  3.0  4.0  5.0  6.0  7.0 


Likert  Scale  Values 


Malpractice  Claims  Filed  per  100  Physicians 


Appendix  Y47 


Frequencies 


Statistics 


MALPRCFL 


N  Valid 

111 

Missing 

1 

Mean 

4.05 

Std.  Deviation 

1.79 

Percentiles  25 

3.00 

50 

4.00 

75 

5.00 

MALPRCFL 


Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  1 

14 

12.6 

2 

11 

22.5 

3 

16 

14.3 

14.4 

36.9 

4 

18 

16.1 

16.2 

53.2 

5 

27 

24.1 

24.3 

77.5 

6 

17 

15.2 

15.3 

92.8 

7 

8 

7.1 

7.2 

100.0 

Total 

111 

99.1 

100.0 

Missing  System 

1 

.9 

Total 

112 

100.0 

Malpractice  Claims  Filed/100  Physicians 

30  T - , 


Likert  Scale  Values 
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Performance  Measurement 
Areas 


Top  Five  Recommended 
Performance  Measures 


Appendix  AB 

Respondent  Recommendations 


From  Part  III 
Questionnaire 


The  Top  Five  Recommended  Performance  Measurement  Additions: 


1)  %  Providers  Satisfied  with  job  or  TRICARE 

2)  %  Staffing  of  PCM  support  staff/positions 

3)  Visits  per  provider  per  day,  weighted 

4)  %  time  providers  at  clinic,  admin,  other  duty 

5)  #  visits  PMPM 


Domains/Performance 
Measurement  Areas 


Appendix  AD 
Gap  Analysis 


Facility 

Type 

Domains 

#  of  Measures 

Johns  Hopkins 

Healthcare  System 

Clinical  Quality 

13 

Workplace  Quality 

5 

-employee  satisfaction 

1 

Patient  Satisfaction 

3 

Clinical  Volume 

7 

Financial  Performance 

10 

Revenue  Recovery 

12 

Cost  Effectiveness 

6 

INOVA 

Healthcare  System 

Clinical  Quality 

11 

Growth  (Marketing) 

6 

Service  Excellence 

4 

Employer  of  Choice 

9 

Physician  Partnership 

5 

Community  Service 

5 

Financial  Health 

22 

United  Healthcare 

Health  Plan 

Financial  and  Budget 

8 

Product  Marketability  and  Pricing 

3 

Operating  Efficiency 

3 

Customer  Service 

3 

Care  Coordination  and  Quality  Initiatives 

2 

Humana 

Health  Plan 

Quality 

15 

Fiscal 

12 

Anthem 

Health  Plan 

Financial 

5 

Performance 

11 

HEDIS 

8 

Customer  Service 

5 

Personnel  Indicators 

3 

Health  Care  Advisory 

Research  Organization 

Board 

-Health  Plan 

Financial 

10 

Utilization 

8 

Clinical  Quality 

7 

Service  Quality 

6 

Internal  Operations 

5 

Innovation 

4 

-Hospital 

Satisfaction 

3 

-Employee  Satisfaction 

4 

Clinical  Quality 

6 

Operational 

11 

Financial 

9 

American  Association 

Research  Organization 

Financial  Development 

5 

of  Health  Plans 

Enrollment  Data 

1 

Utilization 

6 

Liquidity  Tests 

10 

Profitability  Tests 

10 

Leverage  Tests 

7 

iTotal  Civilian  Measures: 

30^ 

MHS 

Quality 

10 

Cost 

15 

Access 

13 

Readiness 

4 

Health  Outcomes 

2 

Special  Interest 

3 

Decriptive  Statistics  Appendix  AC 

by  Organization 

Gap  Analysis 


Domain  Totals  Performance  Measure 


Facility 

Johns  Hopkins 

7 

Totals 

57 

INOVA 

7 

62 

United  Healthcare 

5 

19 

Humana 

2 

27 

Anthem 

5 

32 

Health  Care  Advisory 

Board 

Healthpians 

6 

40 

Hospitals 

4 

33 

American  Association 

6 

39 

of  Health  Plans 

MHS 


6 


47 


Descriptives  by  Appendix  AE 

Domain 

Gap  Analysis 


Civilian 

Organization 

Domains 

Number 
of  Measures 
per  Domain 

Proportion 
of  Domain 
to  Total  Measures 

Quality 

“Clinical 

54 

17% 

“Service 

6 

2% 

“Workplace 

5 

2% 

Total  Quality 

21% 

Financial 

120 

39% 

Operational 

42 

14%  ■ 

Satisfaction 

18 

6% 

-Employee 

22 

7% 

Utilization 

21 

7% 

Innovation 

4 

1% 

Enrollment 

4 

1% 

Population  Health 

10 

3% 

Community  Service 

3 

1% 

Total  Measures 

1  309  1 

1 

MHS  Domains 

Relation  to  Civilian  Sector 

Quality 

10 

21% 

Quality,  Enrollment  and  Satisfaction 

Access 

13 

28% 

Satisfaction/Operational/Community  Serv 

Cost 

15 

32% 

Financial  and  Operational 

Readiness 

4 

9% 

N/A  (possibly  quality/phys  certification) 

Health  Outcomes 

2 

4% 

Population  Health/Preventive  Measures 

Special  Interest 

3 

6% 

N/A  (Data  Quality) 

Total  Measures  |  47 


General  Comparisons 


Appendix  AF 
Gap  Analysis 


General  Comparison 

MHS  Domains  MHS  Private  Sector  Civilian  Organization  Domains 

_ Proportions _ Proportions 


General 

Variance 


Quality 

21% 

19% 

Quality/Enrollment/Satisfaction* 

2% 

Access 

28% 

12% 

Operational/Satisfaction* 

16% 

Cost 

32% 

47% 

Financial/Operational* 

-15% 

Readiness 

9% 

0% 

N/A 

9% 

Health  Outcomes 

4% 

3% 

Population  Health 

1% 

Special  Interest 

6% 

0% 

Data  Quality 

6% 

0% 

1% 

Innovation 

-1% 

0% 

7% 

Utilization 

-7% 

0% 

7% 

Employee  Satisfaction 

■7% 

0% 

4% 

Operational 

,  -4% 

0% 

1% 

Community  Service 

-1% 

*Note:  some  domains  in  the  private  sector  transcended  two  or  more  domains  in  the  MHS. 
Propotions  were  modified  to  take  into  account  those  measures  that  applied. 


I 


Similar  Measures 


Appendix  AG 


Gap  Analysis 

Common  Performance  Measures  between  each  private  sector  organization  and  the  MHS 

#  similar  measures  %  of  total  that  are  similar  Total  Measures 


[Johns  Hopkins: 

8 

14% 

57 

Metrics: 

1 

#  Outpatient  Visits 

2 

#  Patient  Complaints 

3 

Overall  Patient  Satisfaction 

4 

Discharges  per  Day 

5 

#  of  Emergency  Department  Visits 

6 

Total  Drug  Expense 

7 

#  Hours  per  Month  no  Monitored  Beds  Available 

8 

Days  to  Next  Appointment 

INOVA: 

9 

15% 

62 

Metrics: 

1 

#  ofEmergency  Department  Visits 

2 

Cost  per  Bedday/Patient  Day 

3 

#  Visits,  weighted 

4 

FTE/weighted  Bedday 

5 

Overall  Satisfaction  with  Care 

6 

Market  Share  inpatient 

7 

Market  Share  outpatient 

8 

Preventive  Care  Measures 

9 

Health  Status  measure 

I  United  Healthcare: 

3 

16% 

19 

Metrics: 

1 

Total  Expenses  PMPM 

2 

Membership 

3 

Telephone  Response  Rate 

Humana: 

4 

15% 

27 

Metrics: 

1 

Preventive  Care  Measures 

2 

#  Grelvances  per  Beneficiaries 

3 

Pharmacy  Costs  PMPM 

4 

Cost  per  Member 

Anthem: 

6 

19% 

32 

Metrics: 

1  Expense  PMPM 

2  Total  Membership 

3  Membership  per  FTE  per  Month 

4  Average  Speed  to  Answer  Phone 

5  Claims  Inventory 

6  Preventive  Care  Measures 


Similar  Measures 


Appendix  AG 
Gap  Analysis 


Health  Care  Advisory  Board: 


3 

9% 

33 

Metrics: 

1 

#  Patient  Complaints 

2 

Emergency  Department  Visits  ‘ 

3 

Dispositions  per  1000 

-Plans: 

8 

20% 

40 

Metrics: 

1 

Beds  Occupied  per  1 000 

2 

#  of  Emergency  Department  Visits 

3 

#  Enrolled 

4 

Pharmacy  Expenses  PMPM ' 

5 

Outpatient  visits  PTM 

6 

Readmission  Rates 

7 

Physician  encounters  PMPM 

8 

#  Beneficiary  Grievances 

American  Association  of  Health  Plans: 

-HMOs: 

3 

9% 

33 

Metrics: 

1  Enrollment 

2  Claims  Processed 

3  Total  PMPM  Expense 


